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SUMMARY 



The educational decision maker has the responsibility for 
planning for the mediated instructional systems within his 
institution. This planning ultimately results in a decision 
concerning: (A) Whether proposed media systems are justi- 

fied in terms of potential benefits for the investment required, 
and (B) the degree to which presently operating media systems 
should receive continued funding and supports 

To make these decisions in a systematic and objective 
manner, he must have either access to or a means for obtain- 
ing evaluative information and for relating this to the goal he 
wishes to achieve. 

The purpose of this research was the development of a 
strategy for media evaluation and selection. 

The following objectives were accomplished: 

° Definition of media systems as a part of 
instructional technology. 

o Specification of an overall planning strategy for 
the media evaluation and selection. 

° Design of a strategy for identifying the character- 
istics of a teaching task and specifying the media which 
best serve those characteristics. 

o Development of the specifications for an instrument 
(NEW FIELD) which can be used for the evaluation of 
present or proposed media systems. 



INTRODUCTION 



A. Problem 

To improve quality and to extend higher education to more 
students, institutions are investing increasingly in media and 
multi-media systems. At present few guides or procedures 
exist to assist the educator- administrator in planning or evaluating 
such systems and in this absence, costly errors may be made. 
Thus criteria are needed by which an educator- administrator may 
judge what class or category of system, equipment and materials 
are required in order to meet his defined educational goals. 

B. Purpose 

The purpose of this research was to empirically develop 
criteria and procedures to assist in deciding (1) to purchase 
multi-media systems and (2) what variety to purchase in order 
to meet present needs and long-term institutional goals. The 
original request for proposal from the U.S. Office of Education 
called for an analysis of the higher education process which 
results in the development of multi- media systems. Long-term, 
as well as the short range goals of the institution were to be 
considered. The criteria and procedures were to cover all 
classes of educational institutions, but be specific enough so 
that individual institutions might use them in analyzing their own 
situations. Those variables which influence the decisions 
regarding media evaluation and selection were to be considered. 
These include teaching objective, student population composition, 
faculty and technical staff requirements, facilities requirements, 
and cost effectiveness standards. Comparative analyses between 






conv©ntiona.l a.nd. multi- me dia. systems as well as among 
alternative systems were envisaged. 

C . Objectives 

The following four major activities or objectives were 
dentified early in the study: 

1. D efinition: Establish a working definition of 

a "multi-media system." 

2. Basic Planning Steps : Identify and describe 

basic planning steps that may lead to the consideration 
of the use of multi- me dia as an alternative to solve 
specific instructional situations at individual insti- 
tutions. 

Media s e lection : Determine what general types 

of instructional activity (classroom, library, or 
laboratory) can b'£5st be handled by particular multi- 
media systems. 

4* Evaluation : Determine what objective criteria can 

or should be established to evaluate the usefulness 
of multi-media systems in higher educational 
instructional situations. These criteria should be 
practical for the evaluation of the effectiveness of 
a multi- media system at the individual institution. 

Arrangement of this report 

Part I of this report contains expository matter arranged 
according to the four general objectives listed above. 



- Part I, Sec. A, is entitled, "Definition of Multi-Media 
Instruction, " and traces some of the reasoning which led to 
our classification of media systems. 

~ Part I, Sec. B, is entitled, "Basic Planning Steps, " 
and describes a rationale in keeping with the second 
major objective. 

- IPart I, Sec. C, entitled, "Teaching/Learning Strategy De 
scription, " reports on the effort devoted to determination 

of means for matching media to specific instructional 
problems. This is in keeping with the third major ob- 
jective. 

- Part I, Sec. D explains the need and describes the 
means used to conduct the FIELD evaluation. It is 
entitled, "Evaluation of Multi-Media systems." 

thrt II of the report is devoted to the evaluation activities 
during the latter two -thirds of the project. It gives a detailed 
report and documentation of the activities at the American 
Institutes for Research, Washington Office, and in four universi- 
ties where multi-media systems were evaluated. This section 
is divided into four parts. In Section A the "Methodology" is 
described; in Section B the "Findings" of the FIELD tryout are 
presented; in Section C these findings are discussed; and in 
Section D "Specifications" for an improved evaluation device 
(NEW FIELD) are presented. 



I - HISTORICAL DEVELOPMENT 

A. Definitio n of Multi- Med ia In struc tion _ 

1. Mnlti^ medi n, presentations. The classic example 
of a "multi-media presentation" takes place before a large group 
in an auditorium. A two-channel tape recorder is used to program 
and coordinate the sequence of events. One channel is an audio- 
commentary, recorded in advance and often including sound effects, 
music, etc. On the other channel are inaudible tone signals used to 
trigger room darkening and lightening, motion picture, slide and 
filmstrip projector and video tape activating at pre- determined 
moments, as well as programmed halts for instructor intervention. 
All this permits conventional lecturing, question periods, over- 
head projector presentations, etc. 

The foregoing description serves as a narrow, 
functional definition for multi-media presentation. It is indeed the 
type of situation to which the term "multi- media" was first attached. 
However, over the years, this term, has broadened. 

In a recent publication, the Office of Education de- 
fined a new medium by listing the types of electro-mechanical 
devices used to transmit educational information. The list included 
teaching machines, computer assisted instruction, educational 
radio, television, motion pictures, language laboratories, film- 
strips, slides, graphics, audio and video recordings, and devices 

under development. 

In Brown and Thornton’s Higher Education Media 
Study (HEMS), "new media" were classified into approximately 
30 categories and subcategories, one of which v/as "multi-media 
units." Without exception, the particular activities described 
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under the HEMS Category "multi- media units" were physical plant 
facilities especially designed as loci for the type of presentation 
described in the first paragraph. Other categories described 
classes of apparatus which might be included in a physical plant 
facility devoted to "multi-media. " 

If "multi- media" is to refer to something more than 
a general meeting room with a variety of teaching apparatus in- 
stalled, then an attempt must be made to write a useful definition 
which classifies this larger conceptualization. !Many related terms, 
which frequently appear in the literature, impinge to one degree or 
another on our efforts to define the media. Some of these terms are 
listed below: 

"New Media" "Two- Channel" 

"Cross-Media" "The New Technology" 

"Multi- sensory" "Instructional Media Centers" 

Some of the concepts buried among these terms are: 
the concept of newness, the idea of proven instructional validity or 
usefulness, the concept of mixing of instructional means, and the 
physical implementation of these means. 

Obviously, "newness" is a relative matter. For 
example, 8mm film is an old technique (almost as old as radio), 
yet it is considered a relatively new medium in the audio-visual 
field, whereas radio frequently gets left out in considerations of 
educational media. 

The term "media" is often coupled with the words 
"instructional" or "educational" and from this one could infer that 
the medium itself has some educational property. The problems of 
understanding the effect of mediation on the content of the message 



has been elaborately discussed by Marshall McLuhan. F or ex- 
ample, it can be seen that the mediating technique used will 
undoubtedly have some kind of "reputation” or image as to its 
educational appropriateness. 

In a study conducted by Samuel Becker at the Uni- 
versity of Iowa, college students were found to be resistant to 
TV instruction, stating a preference for face-to-face instruction. 
This may have been a reflection of American youth* s concept of 
television as frivolous while college instruction should be serious. 
Or it may have been an expression of a desire for two-way per- 
sonal involvement. 

The need for a defin i tion . For a definition to be 
useful an agreement must be reached between the writer and the 
reader. It must attempt to provide a common understanding. Its 
value is in direct proportion to its explicitness and acceptability to 
both parties. 

One approach would be to examine human learning 
activities and by a process of exclusion, narrow down the field 
until we have defined multi- media instructional systems. A chart 
on the following page indicates how this approach might work. 

The class of "All Teaching/Learning Activities" 
includes everything from table manners to apprenticeship programs 

The class of "Formal Learning Systems" might help 
to zero in on schools and other programmatic efforts to convey 
information or change behavior. 

The class "Media Systems" could include everything 
from the college humor magazine to an instructor's mimeographed 
handout entitled, "Reading Assignments. " 
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HART A - HUMAN LEARNING ACTIVITIES 




LEARNING SYSTEMS 



ALL TEACHING/LEARNING ACTIVITIES 




"Multi- Media Systems" might be defined as any 



intentional instructional activity involving two or more mediation 
elements or devices. But this won't help much because an instructor 
using chalk at a blackboard fits this definition. He is using air as 
the mediation element for his voice and the chalk and blackboard as 
a device for mediating pictorial symbology. 

3. Definition base d orx attributes. A working definition 
might be based on the general attributes of multi-media systems. 

To examine whether a definition listing the attributes of a multi- 
media system would be workable, a "strav* man" can be created, 
and then the fallacies of a defmition listing attributes can be located. 

T he electro-mechanical attribute. Media systems 
usually involve some electrical, electronic, or mechanical appara- 
tus used to enrich an educational presentation. The professor lec- 
turing (audio) in front of a blackboard (visual) can be ruled out if 
the electro-mechanical attribute can be relied on to discriminate 
between systems. The professor has limited his equipment to a 
textbook, some notes and a piece of chalk. Of course, if a video 
picture of the same gentleman were transmitted over some distance 
to students in front of a "tube, " an electro-mechanical device would 
be introduced and mediated instruction would be occurring. 

The electro-mechanical criteria, if used, could 
rule out certain elements normally thought to be part of the new 
instructional technology. Programmed learning material is usually 
mediated through the programmed book format, and as such, it 
qualifies as a non-electro-mechanical form of "mono-media." 

Storage and transmission attribute. A second 
attribute of multi-media instructional system is the capacity for 
storage, retrieval, and transmission of symbolic material which 
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is in some way analogous to real-time human behstvior. For 
example, a phonograph record constitutes a repository of stored 
verbal informationo Sophisticated listeners can almost always 
identify the analog (played- back sound symbology). They rarely 
confuse it with real-time discourse. 

Computer printouts, used in interactive computer- 
assisted*instruction to deliver learning sequences, consists of 
printed symbols wjhich stand for or are analogous to speech utter- 
ances. The student receiving the stored information, which has 
been retrieved from the computer’s memory and transmitted 
electro-mechanically to intout paper, receives through the 
medium of light tri ?.smission and the sensorium of his eyes a 
verbal communication which he must process mentally and react 
to as part of the learning activity. 

Again, the receiver of the information is highly 
unlikely to confuse the computer printout information with real- 
time discourse. Films and television tapes provide for the storage, 
retrieval.transmission and replay of more complex interpersonal 
or natural events. Marshall McLuhan has provided convincing 
argum,ents concerning the energy level required of the beholders, 
arguing that high fidelity playback in any medium demands less 
of the beholder (in the way of synthesizing the experience) than low 
fidelity media which demand the active participation of the beholder 
in order to "build up" a meaningful synthesis of stimulus material 
and stored experience from his own repertoire. 

The storage, retrieval and transmission criteria, 
though useful in a general way, fails to discriminate between 
systems which are multi-media instruction and those of a more 
conventional type. Textbooks and references certainly have storage 
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and retrieval capabilities and one cannot overlook the professor as 
an agent for information storage, retrieval and dissemination., 
Similarly, group discus sion (as an educational medium) depends 
upon multiple storage units and the retrieval and transmis sion 
involved in interaction among units. Yet it, too, does not fall 
within the framework of multi-media instruction. 

It may be that the field of multi- media is helped 
toward definition by inclusion of the storage- retrieval-transmis sion 
attribute. All of the media tend toward linear or sequential de- 
livery of information under the control of the author or editor of 
the original storage document. Even random a,ccess devices are 
under the control of a linearizing program when they are used for 
educational activities. 

It is important to note that random access computers 
and display equipment (microfilm, etc. ) can be used as educational 
tools if programmed in some meaningful sequence dependent upon 
the response activities of the learner. These same devices are 
also used for research where the same storage- retrieval and 
transmission activities occur without the intervention of a programmed 
control exerted by an editor. 

c. Multi-se n sory attribute. The classical multi- 
media presentation impinges on the human ear via the use of audio 
tape and upon the eyes via projection images observed on a screen 
or video tube in two dimensions. The captioned film for the deaf 
program involves the use of conventional sound films with support- 
ing titles, sometimes projected by a separate projector kept in 
synchrony with the sound film projector. But we would not wish to 
rule out the "captioned films for the deaf" activity sirnply on the 
basis that it did not cross sensory boundaries. 



Ruled out might be the use of transparencies, 
filmstrips, and slides on the basis that the perceptual channel 
being used was the visual channel. But, the vast majority of slide 
presentations, filmstrips and overhead transparency sets are ac- 
companied by guidebooks, narration or other audio capabilities 
which augment the pictorial imagery. Still, an art teacher, lec- 
turing from slides alone, might not qualify as a multi- media 
instructional system since the only electro- mechanical medium 
involved is the projection of visual images. 

We might also examine the amplification of 
normal sensory abilities through the use of media systems. In 
C3.SO wc! must find a way to discriminate between optical and 
television microscopy. Optical microscopy is old-fashioned, well 
accepted and of high fidelity. The other is new and classified as 
a media-based system. TV cameras frequently can provide mag- 
nifications far exceeding that of the human eye and zoom capabilities 
which are physically impossible for live observers. 

d. Distance reduc tion attribute . In addition to the 
amplification of information intended for sensory absorption, 
multi-media systems frequently permit the extension of the senses 
geographically over long distances. The obvious examples are 
live, closed and open- circuit television. Dial Access retrieval 
systems, computer information storage and access, blackboard 
and radio. Here the medium makes it possible for the long-distance 
examination or study of a subject by students, often at an increase 
in fidelity or observational acuity. 

e. Intimacy attribute. Multi- media instructional 
systems frequently make for a heightened sense of one-to-one 
instruction between instructor (or rather his reconstituted analogue) 
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and the student. Video-tapes, language labs, audio-tutorials, 
self-instruction and programmed-instruction materials, and 
computers operating in an interactional mode can m.ore often 
achieve a heightened sense of personal interchange, (even though 
the medium interferes with the actual warmth of direct personal 
contact) than could be obtained through conventional large-group 
classroom instruction. 

f. Reality changing attribute . Media systems 
normally alter the reality they seek to reconstruct. That is, the 
delivered analogue is perceived as different or "less whole" than 
the real educational event one perceives through instruction. 

Teachers of reading in center city schools insist 
that children must have experiences in the real world before they 
can " read" about experiences in textbooks. They make the point 
that children who have not had a trip to the farm or the zoo cannot 
attach meaning to the printed symbol cow'or the pictorial repre- 
sentation of a zebra. Clearly, then, the reconstruction in the mind’s 
eye of a mediated educational interaction requires a base of ex- 
perience in order for the student to make any sense at all out of 
the transmitted analogue. 

In a very real sense the act of reading, studying 
the illustrations, re-reading, and taking notes from the text is a 
partial reconstruction of a lecture and blackboard style of educa- 
tional presentation. Authors of texts commonly prepare a text in 
a third- per son, formal format. 

Although the textbook is less than real in the 
sense that it is not alive, no voices can be heard or images be 
seen, it is also "m ore than real " in the sense that it can be read 
at an individual pace anywhere and anytime as many times as 
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desired. That is, reality is lost by converting lectures to textbooks, 
but a new dimension of storage, retrieval and transmission as well 
as "instant replay" is added by the conversion of a lecture series to 
a textbook. 

An example of the ability of media systems to 
alter reality may be seen in the use of time lapse photography to 
demonstrate heliotropic reactions in teaching botany. 

4. Towa rd a pragmatic classification. How then can 
media systems be classified or organized into meaningful clusters 
or styles or mediation? How can 30 or more categories of educa- 
tional media systems be coalesced into some meaningful arrange- 
ment ? 

In order to delimit our study and to identify the 
specific types of systems to be studied, we examined some of the 
salient features. of the various media systems. On the following 
page is a chart entitled "Dichotomies which aid in Media Classifi- 
cation". These, and many other issues, were mulled over in the 
process of sorting media systems into groups. 
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CHART B - DICHOTOMIES WHI CH AI D I N MEDI A CLASSIFICATION 

Verbal Format 



PRINT 



VS 



AUDIO 



The message is converted 
to printed characters, than 
"read" (rather than heard). 
(Program.ed Text) 



The spoken message is 
recorded, stored, re= 
constituted and "heard" 
(Language Tapes) 



Location 



LOCAL 



VS 



DISTRIBUTED 



Storage and retrieval 
are accomplished at 
or near the learning site. 
(Classroom Filmstrip) 



Storage, retrieval and 
transmission are 
centralized. 

(CCTV applications) 



Availability 

DEMAND VS 

Instruction occurs as called 
for by student(s) or teacher. 

(DAIRS) 



SCHEDULED 

Instruction pre-planned; 
Students & teachers fit 
their plans to master 
schedule. (ETV) 



Intimacy 

REMOTE VS 

Instruction has formal, 
stylized quality; 

"involvement" is low 

Definition 

COOL VS 

Medium transmits only 
essential stimuli; students 
supply "fill in" from 
experience; dema.nds atten- 
tion. (Black board -by -wire) 



CLOSE-UP 

Instruction has a personal, 
intimate or "real" quality 



HOT 

Rich detail is provided; 
learner is passive observer 
"feelings" rather than 
"intellect" reached. 
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Attempting then, to take various aspects and 
attributes of media into account, a system of categories which 
permit the clustering of media systems has been created. The 
arrangement of systems into classes has utility for planning 
evaluative measures^ 



The five classifications arrived at make possible 
the categorization of all the major types of media systems. They 
are also somewhat related to conditions of learning or teaching 
learning strategies. In addition, they lend themselves to categori- 
zations according to instructional mode. 

5. Class es of systems to be eva luate d. The five general 
classifications of multi-media are identified below. Various attri- 
butes were used to separate systems into classes. Two classes were 
set up under a "verbal linear attribute" rubric because they are 
essentially verbal linear media; two have been identified by student 
activity qualities. 

a. Verbal linear attribute . "Let us Reason 
Together. " The two classes defined as verbal mediated have been 
arbitrarily entitled "Print-Structured" systems and "Audio- Linear" 

systems . 



(1) Print- s tructured . Within the cj.assification 
Print- structured are CAI (computer assisted instruction), PI 
(programmed instruction), and ERE (Edison Responsive Environ- 
ment). Other mediation systems depending in large part on printed 
digital verbal stimulus presented sequentially^ according to planned, 
linear thought processes would fall into this group. 



(2) Audio- li near . The Audio-linear cluster con- 
tains language laboratories. Dial Access Information, Retrieval 
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Systems and Tele-lecturo apparatus. These and the more con- 
ventional audio teaching systems (like phonograph records and 
radio) arc set aside as a class of systems based on the delivery 
of audible, iconic verbal information and stimulus materials in 
a planned sequentia l pattern. 

b. Multi - media attribute. "Let me show you. " 
Another media family, which consists of si m.ultaneous audible and 

visible portions, has been separated into two clusters identified by 
distribution mode, distance, and storage location, 

(1) Local. The "local multi- media family" 
includes multi- media classrooms, audio -tutorial tape/slide pre- 
sentations, and local and librairy-based audio-visual systems in 
general. Where the teaching materials are located at or near the 
traini n g site and used at the option of the student or instructor, 
then the media system will be considered to fit into the "local 
multi- media cluster. " 

^ 2 ) Distributed . The second cluster of the multi- 
media family systems is the electronically distribured media 
group. This includes CCTV (closed- circuit television), ETV 
(educational broadcast television), DAVid (Dial Access Video), 
and B-B-W (blackboard- by- wire). Other systems which permit 
delivery of both vis ual component and audio signal over large 
distances will fall in this class. 

C’ Activity mode. A final cluster or family of 
new educational techniques or systems is that group which can 
best be identified by its heightened levels of student activity. 
Examples of this class are: the self-confrontation uses of the 

VTR (video tape recorder), the "wrap-around" environments 
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(like the classroom simulator and the driver trainer), the various 
games, interpersonal discussion experiences, and devices which 
have an open-ended (non-linear) activity as their reason for being. 
The important factor in these devices is the c omplex and opem 
ended fo rm of behavior or activity on the part of the student. In 
most cases the student activity in thi s group of mediated instruc- 
tion approaches more closely the final terminal behaviors sought 
by the instructor than do the artificial ''test" behaviors associated 
with more "academic” objectives. 

It should be pointed out, of course, that print- 
ed and audio— linear systems as well as the local multi- 
media and the distributed electronic media all permit one form or 
another of student activity. All too often student activity is limited 
to paper -and- pencil responses. Other systems may include push- 
button choices, tele -typewriter interactions, or tape-recorded 
utterances (as in the case of language systems). But each of the 
aforementioned response systems has a closed, set of possible 
responses while the ACTIVE MODE systems permit a much larger 
repertoire of responses to be emitted. 

Appropriate uses of media system classes . The 
above mentioned media system classes have been set down in a 
chart on the next page. Major members of each class are entered 
at the top of the chart. A listing of various instructional modes 
appears a.t the left-hand side of the chart. 

Where instruction can effectively be carried out 
by the media system, an "X" has been entered opposite that 
instructional mode classification. 
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I - HISTORICAL DEVELOPMENT 



B. Basic Planning; Steps: A Model for Media System Planning . 

1. Objective . The second objective of the program for 
evaluation of multi- media systems in higher education was the 
development of a planning paradigm for use by higher education 
institutions. 

Planning of the type common in industrial and military 
establishments is rare in American colleges. Most new programs 
are developed on a "vest pocket basis. " An aggressive, dynamic 
school or college administrator is selected and employed to "start- 
up" a new program, activity, college, etc. During the develop- 
mental stage, he works "out of his vest pocket" because he is never 
in his office; he is running too hard. 

The essence of this method is that one main's dreams and 
ideas about the system under development constitute the plan. His 
authority and quick, expeditious decisions serve to bring the new 
activity to fruition. 

In the following paragraphs a procedure is described for 
the more planful and orderly implementation of a multi- media 
system, 

2. Planning Model. A general procedure is described 
here for the use of planners and decision-makers in the field of 
Higher Education. It is a theoretical model in the sense that it 
provides a hypothetical set of steps to be followed by non- media 
oriented educators and administrators in developing new educa- 
tional systems plans. It is a pragmatically useful device only insofar 
as it outlines a series of practical steps to be taken by real persons 



in preparing feasible programs for actual implementation in col- 
legiate instructional settingSo 

It attempts to clear away some of the mystique as- 
sociated with planning for and securing support for media. In its 
place a relatively simple and straightforward procedure is offered 
which will assist educational administrators and multi-media 
planners to successfully apply new media to the educational ob- 
jectives of their institutions. 

The procedure consists of six major activities with a 
number of sub- routines to accomplish specific ends. The major 
activities are as follows: 

IDEATION 
FORMULATION 
PROPOSAL DEVELOPMENT 
SUBMISSION AND FUNDING 
IMP LE ME NT A T ION 
EVALUATION 

The study of the flov/ chart on the next page discloses 
the three additional sub- systems: 

Funding Resource Search 
Feedback Cycle 
Report Cycle 

Within each of the blocks of the systems diagram a 
number of procedures should be followed. On the following pages, 
descriptions of these activities are given. 

a. Ideation . During this stage of planning for 
the acquisition of a multi-media system, the educational innovator 
should follow a systematic approach in the generation of ideas. A 
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considerable body of literature has been developed which could 
assist him in generating ideas. Most of the current work is being 
done in a relatively few centers. Notable are the studies in Oregon 
at the Center for the Advanced Study of Educational Administration 
(CASEA), in Ohio, Strategies for Educational Change (SEC), and 
through the' National Training Labs' Cooperative Project for Educa- 
tional Development (COPED). The steps described in the following 
paragraphs are the simplified synthesis of the concepts described 
in these studies. Four basic sub-systems are involved. These are 
shown in the chart on the next page. 

(1) Problem definition. Ideas for multi-media applica- 
tions usually arise in the context of some form of ongoing educational 
activity or problem. Some examples of educational problem situa- 
tions which initiate action toward study of a multi-media application 
are (a) an increase in the number of students requiring a restruc- 
turing of the curriculum; (b) decrease in the number of skilled staff 
members which makes the use of media the only plausible means 
of employing staff members at the highest level of skills (tutorial), 
and (c) advancements in information technology which force an ad- 
aptation of the teaching program to the newer approaches. In any 
event, most new ideas arise out of need experienced in the educa- 
tional context. 

The planner should first define the problem which he is 
seeking to solve with the media application. The media innovation 
the planner has been thinking about may point to the prob l em for 
which it is offered as a solution. At this stage it is the proble m 
which must be identified, rather than its solution. Premature 
identification of a solution will stifle a full analysis of the problem 
and the specification of a complete set of alternative solutions. 
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The definition of the problem will normally go through 
a series of steps which involve increasing levels of specificity. 
During the process of defining the educational problem a good deal 
of effort should be devoted to discovering the real root problem 
and assigning a relative degree of importance to its solution. In 
fact, during the initial process of definition a number of subsidiary 
problems may be defined. This may in turn require an examina- 
tion of the general and specific goals of the program. An estim.ate 
of the present degree of success in meeting those goals may be an 
additional by-product. While analyzing a high drop-out rate in 
engineering curriculum one may reveal another problem concern- 
ing the inadequacies of students completing an introductory 
mathematics course. Collecting data about this problem may help 
specify the number of students who are inadequately prepared for 
the engineering curriculum. 

(2) Sea rch for alternatives. After the problem has 
been clarified, a search for alternatives should be instituted during 
which the original multi-media idea should be re-examined. Other 
alternatives which may suit the problem should be sought. The 
existing teaching-learning system may well be considered as an 
alternative. Other possibilities which give promise of success in 
dealing with the problem may have emerged during the "problem 
definition stage. " All should be described in some detail. 

Some of the elements which should be included in de- 
scriptions of the alternative solutions would be: (a) a detailed 

statement of the proposed solution, (b) inadequacies of the proposal 
in terms of those criteria for a good solution for which the par- 
ticular solution falls short of the optimum, (c) elements of new 
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problems which will be created by the solution, (d) anticipated 
requirements for staff skills, teaching materials, equipment, 
space, costs, etc., (e) time required to implement, etc., 

(f) advantage of the proposed solutions, (g) side benefits received 
from the solution. 

The extent to which the alte rnativevS can be detailed in 
a readily comparable format will help in the later stages of plan- 
ning. 

(3) Criteria establishment. At the same time that the 
educational administrator is detailing a set of alternative solutions 
to the problem, he should be setting criteria v/hich will permit him 
to select among the alternatives. This implies a careful study of the 
constraints which are operating. The criteria should specify the 
quality, quantity, and appropriateness of the various outcomes 
which might be expected from alternatives offered as solutions to 
the problem. 

This activity should go on simultaneously with Step a. (2), 
"Search for Alternatives", and has therefore been shown on the at- 
tached flow chart as a parallel path. The product of Steps a. (2), 
Alternatives, and a. (3), Criteria, will be brought together during 
Step a, (4) below. 

(4) Selecting a Iternatives. Given the set of detailed 
alternatives and the means for making the selection, the educational 
administrator opts for the alternative which gives the best general 
fit to the criteria. This selected alternative will be subject to re- 
view at a later date during the general Step b. , "Formulation of 
Plan. " For the present, this selection is made in order to pro- 
vide a basic plan of action upon which the "Formulation" stage can 
be initiated. 



b. F ormulatiorio The formulation of a more 
complete multi-media systems plan involves a number of steps. 
Among these are a review of the literature covering the specific 
media system (or category of media system) that has been select- 
ed. Based on this literature review a visitation program can be 
planned after which a system configuration study based on the home 
institution’s peculiar needs can be conducted. 

This study will in turn permit a preliminary designa- 
tion of appropriate components. Designation of equipment, staff, 
space, materials and other elements will permit a rough-cut 
costing of the system. A tentative timetable for installation and 
implementation of the system should also be prepared. These 
documents then become the vehicle for a re-evaluation of the 
selected alternative in terms of the original criteria developed 
during the ideation sub- routine. 

A strategic step in the formulation phase is the develop- 
ment plan for obtaining institutional approval for the concept under 
consideration. Each of the elements of the formulation sub-routine 
which have been identified above are shown in the chart on the next 
page and are considered in greater detail in the paragraphs follow- 
ing. 

(l) Literature review . A thorough review of the lit- 
erature should be performed. This review will produce several 
bibliographies. These bibliographies should be broken down by 
class of systems (for example, DAIRS, ETV, CCTV, VTR applica- 
tions, CAI, PI, LL, etc. ), and the various sub-classes within a 
system should be identified where logical separations of the litera- 
ture can be made (for example, between ETV and CCTV). 
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(2) Visitation program . After the review of the litera- 
ture is complete, a visitation program should be initiated. The 
formulation phase should include the selection of an appropriate 
number of sites which have some comparability with the home in- 
stitution. 

The use of an instrument (FIELD) discussed later on 
will guide the information gathering effort. 

The results of the visitation program should be drawn 
up in a document which lists the various pitfalls to be avoided, the 
special ingredients or elements to be considered, and the new variables 
which have been uncovered. This report and the literature re- 

view wai become the basis for Step (3), "System Configuration 

Study. " 

(3) System configuration study. Upon completion of 
the visitation program and its resulting report, the educator 
should make a first attempt at specifying a system appropriate to 
the needs of his home institution. This will require a review of 
the product of paragraph a. (1) above, "Problem definition, " and 
the product of paragraph a. (3) above, "Criteria for selection of an 
alternative. " A further examination of the specific requirements 
in the home institution will be required at this point. Such a study 

might consider some of the following variables: 

(a) Space (new construction or existing space) 

(b) Staff 

(c) Specification of function 

(d) Materials specification 

(e) Use levels 

(f) Temporal requirements 

(g) Expansion or conversion, etc. 



(4 ) Preliminary designation of system components. 
This step can be carried out only after completion of the general 
configuration plan. This step involves the writing of specifications 
for a facility, if a facility is part of the plan. It is during this 
phase that electronic and mechanical components as well as human 
and information components will be identified in an appropriate 
degree of detail. 

Examples might be: the selection of 8 mm over 16 mm 
projection equipment, the selection of vidicon TV camera and 
equipment vs. orthacon cameras, or the choice of cartridge load- 
ing tape decks over reel-to-reel models. Specific brands and 
models would not be identified at this time, but it would be expect- 
ed that occasionally the functions or operations specified would be 
available only in the products of a single, specific manufacturer. 

(5) Rough-cut costing and timetable. Based on the 
preliminary designation of system components a first effort can 
now be made to establish unit costs for the elements of the system. 
The basic categories for costing are as follows: 

(a) Equipment (the hardware component) 

(b) Staff 

(c) Materials (information) 

(d) Space 

In each of the foregoing elements of cost a specific 
methodology was devised for the use of educators. In each case 
the methodology permits the full elaboration of appropriate 
schedules to indicate rough-cut costs. (Cost estimate is part of 
the FIEL D; see p. 69 and ff). 

(6) Re - evaluation of selected alternatives. At this 
point, the system specification, costs, and timetable should be 



re-evaluated against the original criteria developed during the 
Ideation sub -routine (par. a. (4). In all likelihood the admini- 
strator or decision-maker will now be considerably more 
sophisticated in his application of the selection criteria that were 
generated in the early stage of the program. He may well have 
improved the criteria or reconsidered the original set of alterna- 
tives, In any event, the design should be subjected to a rigorous 
analysis at this point to determine its appropriateness for use in 
overcoming the original problem. 

A departure of significant magnitude should be con- 
sidered a warning against further effort, A full review and reso- 
lution of problems should occur before going on. 

(7) Institution approval. The system has been elabo - 
rated in sufficient detail to permit its evaluation by faculty groups. 
Trustees and possible sponsors should also be permitted to study 
the plan which has been formulated. 

If a single most important part of the Basic Planning 
Steps for Multi -Media Acquisition could be identified it might well 
be the plans for obtaining in-house approval. The number of 
excellent plans which have died in the faculty senate or molded 
away on the vice-president's desk are legion. But this very fact 
points the way to a gold mine of research information concerning 
strategies for obtaining approval. A careful, and possibly some- 
what covert, examination of "past performances" should be under- 
taken. Learn as much as possible about past ideas (both suc- 
cessful and unsuccessful) so that an artful plan can be devised. 

Some of the elements which should be considered in 
designing a strategy for obtaining approval are as follows: 
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(a) Combified use . Can allies be found for your 
scheme by offering the shared time use of the facility or system 

to other departments, programs, etc.? 

(b) Alternative channels . Are there several 

"routes" to approval? 

(c) Neutralization of threats. Can the engender- 
ing of negative reactions be avoided by careful analysis of the 
prerogatives, expectations, and aspirations of possible protago- 
nists? (This step, the careful search for objections to the plan 
and the incorporation of modifications to circumvent or limit the 
intensity of objections may be the single most important element 

of the in-house approval cycle. ) 

(d) Funding rrlternatives. Effort should be de- 
voted to determining one or more possible sponsors for the pro- 
gram and identifying the requirements each will impose, on the 
program. 

It should be remembered that acceptance of the multi - 
media concept is all that is expected at this in-house approval 
point. A further step will be described (after the proposal develop- 
ment cycle) which involves formal submission and the real ap- 
proval which is evidenced by funding. 

Part of the planning of strategy for approval should 
include a means for obtaining material evidence of the approval 
in the form of institutional support for proposal development acti- 
vities. This need not necessarily involve very many dollars (for 
pilot programs, consultants, tests, etc. ) but should include com- 
mitment of the time of specialists within the university or college 
hierarchy who can assist in obtaining the necessary information 
and preparing documents for inclusion in the proposal. Distribu- 



tion of the load to others within the institution lightens it for the 
innovators while at the same time enlisting the positive support 
of a broader group of persons involved in the effort. 

(8) Sponsorship study . Given approval by the trustees, 
faculty senate, dean, department head, or other approving autho- 
rity within the institution; the initiator of the new media acquisi- 
tion plan should press on to the next Phase - Development of a 
Proposal. It is well, however, to re -consider the possible fund- 
ing resources or sponsors before too much effort has been de- 
voted to the actual proposal. Therefore, a separate funding re- 
source search has been indicated on the flow chart and will be 
described in the report. 

A variety of funding agents exist. Perhaps several 
different federal government programs can be called upon to join 
in support of all or part of the system. Each may require dif- 
ferent elements in a proposal. Frequently state and local govern- 
ments can contribute funds for an equipment acquisition program 
if the proposal meets their specifications and demonstration needs. 
Foundations are often important sources of early funding. Fre- 
quently, they have special requirements in terms of the originality 
of concept or uniqueness of functions that will be provided. For 
these reasons the sub -routine of obtaining information concerning 
funding requirements should be initiated at the same time that the 
proposal goes into the developmental stage. Provision should be 
made to incorporate these learnings during the actual Systems 
Design stage. 

c„ Proposal development. If a program or pro- 
ject survives into the proposal development stage the initiator of 
the innovative multi -media system acquisition plan should be able 



to count on institutional resources for assistance. Ideally the in- 
house approval (obtained in paragraph b. (7) above) will include a 
commitment on the part of the department head, college dean, 
faculty, or trustees for institutional investment of time, energy, 
expertise and sometimes even money in the task of proposal de- 
velopment. At this point a larger team can be brought into the 
program. The information gathered during the Ideation and Form- 
ulation Phases can now be re -consider ed, analyzed and reworked 
into a more coherent proposal. The various staff members who 
will have operating functions during the implementation and evalua- 
tion stages should now be brought into the program to make their 
more specific contributions to the actual proposal effort. 

This effort consists of a relatively diversified set of 
tasks which elaborate upon the original design. These various 
tasks can best be performed by specialists in each area. The 
proposal will not have an authentic quality without the direct parti- 
cipation of specialists in the various sub-fields. 

More importantly, it must be a sound, uncomplicated 
statement (or commitment) on the part of these same specialists of 
the tasks they will perform (the part they intend to play) in the 
overalTacquisition and implementation program. 

For this reason, it is essential to involve (in addition 
to the administrator or decision-maker who has initiated or co- 
ordinated the original program) members of the following specicilty 

Educational Process Specialist (Educational Psychologist); 
Educational Materials Specialist (Editor /Graphics) 

Subject Matter Specialist (Professor); 



groups : 
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Facilities Specialist (Architect or Building and Grounds 

Specialist); 

Finance Specialist (Business manager); and 
Personnel Specialist (Staff recruiting). 

An important additional group of practitioners must also be brought 
in at this time. These are the sales personnel for the various 
profit and non-profit organizations who may contribute to the over- 
all program design. Of course the salesmen for textbooks, tapes, 
projectors and computers have an obvious part to plan in helping 

to define the components they could offer. 

More obscure, but equally important, are the non- 
profit or consulting resources which may also be needed. An 
example is the consulting engineer who renders professional 
judgments concerning configuration of equipment components. 

Many multi -media acquisition planners will consider the appoint- 
ment of an appropriate evaluation contractor (who can be expected 
to take an independent view of the program and render a judgment 
without fear or favor). The proposal development routine is shown 
in the chart on the next page. 

^1) Problem statement. During this portion of the ef- 
fort a clear and amplified restatement of the problem (defined 

earlier in par. a. (1) is written. 

(2) Approa ches. The various possible approaches 

(called alternatives) to the problem are restated. (The continua- 
tion of conventional instruction should be offered only if it is 
among the viable solutions to the problem). 

(3) Benefits. The benefits of each of the approaches 
are identified and compared in as much detail as seems appro- 
priate. 
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(4) System design. The selection of an appropriate 
approach is restated in considerable detail, giving the rationale 
for the selection of that approach and its benefits. Having given 
the rationale for the selection of this alternative, the proposal 
should then offer a detailed description of five basic elements 

of any system design: the proposed student body , the staffing, 
the space requirements, the equipment package, and the teaching 
materials procurement plan . 

(5) Costing s ection. A section of the proposal should 
be included which clarifies and extends the costing developed 
during b. ( 5). Accounting practices, procurement plains, etc. 
should be covered. 

(6) Timetable. In order to make clear the specific 
personnel and start-up problems which can be expected to occur 
during the course of an implementation cycle, a timetable should 
be developed which gives such details as the following: 

° Student use t.ables showing input -throughout - 
output; hours of use, etc. 

° Manpower loading during planning and design 
stages, start-up stages, and the steady state manpo^wer expecta- 
tion. 

o Space requirements and plan as the program 
grows (be sure to match with student growth expectation) 

° Equipment phasing table showing when various 
elements of equipment will be required and the accumulated costs 
of same (be sure to also show equipment attrition, maintenance 
costs, margins for spares, etc. ) 

o Procurement chart showing the order placing 
dates, payment dates, etc. for the acquisition and/or development 



of appropriate materials for use in the system. The cost (in man- 
power) for local development of teaching materials should be ref- 
lected in the staff manpower loading charts given above. 

(7) Funding plan . A complete funding plan should be 
developed at this time. Institutional resources, alumni funds, 
special grants from foundations and local, state and federal con- 
tributions should be identified. Special conditions of lending in- 
stitutions should be explicitly stated. The timetable of funding 
requirements should be backed up by adequate documentation of 
the "if-then" contingencies in the funding procedure. It is im- 
portant to let each organization that will participate in the funding 
of the multi -media system know exactly what other group>s and 
contingencies are involved in the funding procedure. 

(8) Risk of refusal. It is well to indicate in the pro- 
posal some of the expected outcomes for second and third alterna- 
tive approaches which would be used in the event that the proposed 
multi-media system is not acquired and put to use. These often 
can be best stated as expected expenses which will be incurred if 
the proposed effort is not carried out. Expenses are not always 
financial. A severe and progressive weakening of a language or 
engineering program can best be demonstrated if it is compared 
to the program of a neighboring institution where the language labs 
or computer assisted instruction in engineering are demonstrably 
effective. This can be a persuasive item of expense which an in- 
stitution may incur if it does not move forward along the new edu- 
cational technology path. These pragmatic losses or anticipated 
risks should be well-documented in a proposal. 

d. Submission and funding cycle . Given an ap- 
propriate, accurate and persuasive proposal, the next step is the 
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carefully planned submission of the proposal. Already time will 
have been devoted to the study of funding resources. This search 
is normally conducted by the administrator among alumni groups, 
government agencies, or other resources. To one degree or 
another this effort will shape the proposal (as has been discussed 
in par, c. above. It will also, of course, influence the submission 
and funding cycle. The Submission and Funding Cycle is shown in 
the chart on the next page. 

Many colleges and universities have a ’’development 
office, ” an "alumni relations office, ’’ or a research and grants 
administrator. Wherever possible these specialists should be in- 
volved at the earliest possible stage. Their advice, counsel and 
review should be sought during the stages identified in Phases a, 
b. , and c. above. But their special skills come into play most 
pronouncedly during the submission and funding cycle. 

The attached flow chart indicates several important 
subphases: groundwork, submission, call-back routines, reject, 

rework, and acceptance. Each of these phases is an important 
element of the submission and funding cycle. 

(1) Groundwork . Field calls on each of the possible 
funding resources and careful study of their requirements and ap- 
plication forms should be undertaken well in advance of the sub- 
mission of the proposal. The investigation should include research 
into the timetable which funding agencies follow as well as the 
amounts of funds available. The expectations and decision-making 
criteria of the personnel involved in making the funding decisions 
should be understood. The more complete the planning and execu- 
tion of a submission and funding cycle becomes, the greater are the 
chances of acceptances and implementation. Advance notice of 






39 



intention to submit a proposal often helps. Pre- submission con- 
ferences based on draft documents may disclose repairable faults. 

A receptive understanding of unwritten aspects of the problem can 
often be engendered. 

( 2 II Submission. The proposal is offered to the sponsor- 
ing agencies with a short covering letter requesting favorable action 
and reciting, in simple, compelling terms, the merits of the pro- 
posal and the risks involved in its refusal . 

(3) Call- back routine. It is not sufficient to simply 
offer a proposal to a prospective sponsor. One must also be per- 
suasive in delicately calling his attention to the need for action. 

The unsubtle attempt, however, to apply pressure of some sort 
frequently has the effect of aborting the mission. Therefore, the 
creation of a delicate sense of urgency without the implication of 
threat or pressure often has the desired effect. During the plan- 
ning phase it is well to work out a number of "excuses" for main- 
taining contact with the persons along the decision tree where a 
funding decision will be made. 

(4) Rejection of proposal . From time to time a pro- 
posal meets unfavorable reception. Usually several options re- 
main open. And one significant strength has been added: that of 

experience. 

During the reaction to the rejection of a proposal, a 
very careful effort should be undertaken to determine what parts 
or elements of the proposal (or what funding limitations of the spon= 
soring agency) resulted in the denial of the application. Frequent- 
ly, such information is useful in re- writing the proposal for sub- 
sequent submission. In any event, all information on the reasons 



for rejection should be passed along to the proposal writing team, 
to help during rework of the prop-^sal in a second version for sub = 
mission to a new sponsor^ 

The possible receipt of rejection evidence should be 
considered during the original planning. Appropriate cut-off dates 
and withdrawal dates should be determined. After a proposal has 
been on the sponsor's shelf for a certain length of time it should be 
subject to withdrawal and submission to another possible sponsor. 
Wherever possible the specifications for re-work of the proposal 
to meet a second sponsor's needs should be developed in advance. 
This way the changes necessary for a new sponsor can be initiated, 
even though the originator is reeling from the impact of rejection. 
(Having a plan or a useful course of action at this point in time 
often results in re submission to the same or different sponsor. 

Selection of a second sponsor is an important sub- 
phase of the rejection activity. If an effort has been made to 
identify alternative sponsors during the sponsorship study (identi- 
fied in d. above) , then, it may be possible to resubmit the pro- 
posal to a different sponsor almost immediately. 

(5) Acceptance. Acceptance of a proposal by a funding 
agency should be the signal for the direct and rapid implementation 
of the early steps envisioned in the proposal. Several checks, how- 
ever, should be carried out before moving too hastily. Some of 
these are: 

(a) Wait f o r a contract: It is foolhardy to commit 

money (or staff energy) to a program without, at the very minimum, 
a "letter of intent" which describes the sponsoring agency's degree 
of commitment to the progrcim. 
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(b). System review: Wor d of acceptance should 

be followed by a very careful system review incorporating all the 
personnel mentioned under c. above, and re-studying the 
fea Sibil' y of doing that which has been proposed. If the sponsor 
has imposed new conditions, reduced the dollar value of the grants 
or otherwise placed significant qualification on the proposal, then 
the proposal team should re ^adjust their plans to meet the new 
needs. 



During this phase a series 



of assignments and follow-up checks concerning the different res- 
ponsibilities will be carried out. The central responsibility is, 
quite obviously, an administrative and management task. In order 
to carry out these operations college administrators may require 
some assistance in the form of generalized checklists of events 
and types of activities which must be undertaken. Rather than 
detail here the fairly well-defined management techniques, we 
will simply indicate several paragraph headings and the possible, 
content for that paragraph: 

(1) Assign monitor or program manager 

(2) Establish responsibilities (within institution and 
outside agencies and constraints) 

(3) Determine priorities 

(4) Plan accounting techniques and methods 

(5) Initiate staff search, recruiting, etc. 

(61 Initiate PERT or CPM planning and review methods 

(7l Establish a technique for reporting and feedback 

If 

to sponsor 

Wqrk out a dry-run plan and criteria for determi- 



nation of its effectiveness 



(9) Develop scheme for emergency decision-making 
and plan-change authority 

(10) Attempt to predict typical pitfalls for wet-run 
operation and provide solutions 

fo Evaluation . The apparatus for evaluation 

v/ill have been simply and clearly developed during stage a. (3) 
(Criteria establishment), during stage b. (3) (in which the system 
configuration study considered the requirements of the whole 
institution), and in phase c. (3) of the proposal development 
routine during which the expected benefits of the system imple- 
mentation were defined. 

Unless a carefully designed evaluation mechanism is 
followed, the rest of the educational community will at a loss to 
determine whether or not the innovating insjtitution has found a new 
educational tool. Thus the evaluation subsystem should refer back 
to the following elements and carry out steps or programs as indi- 
cated below: 

Criteria design 
Observation methodology 
Effectiveness measures 

Units or quantitative increments of benefits 
Cost accounting reviews 
Cost per unit of benefit 

The cost per unit of benefit mentioned above will be 
seen to be a difficult element to incorporate in a system for plan- 
ning multi-media acquisitions in the field of Higher Education. 
However, a first step should be taken toward the involved and arduous 
data collection activiti^ needed to carefully assess the cost and 



effectiveness of instructional systems. If this way of stating 
the case for cost effectivness seems too harsh for academicians, 
let them remember the taxpayer. The taxpayer’s dollars are 
becoming a larger and larger factor in the educational funding 
equation. Careful analysis of every college's expenditures is 
expected. As the gap widens between the public’s expectations 
and the ability of colleges to perform this analysis, colleges 
can expect intense scrutiny and quite pos ably some severe 



criticism of their efforts. 



I » fflSTORICAL DEVELOPMENT 

Go Teaching/Learning Strategy Description 

This section focuses on a critical process in the over- 
all educational planning activity » An assay of the broad objectives 
of the media system will identify the objectives of the instructional 
units which comprise the educational systems A process lor 
identifying these objective Sg analyzing them and specifying the 
media elements which promise to serve these objectives is de- 
scribed here, 

lo Stateinent of the problems Educational planners 
and decision-makers at all levels need systematic procedures to 
collect and organize information for instructional design^ as well 
as evaluation techniques for continuously assessing the quality 
of their decisions. 

In the specialized area of instructional media 
selections educators are particularly in need of carefully de- 
signed and proven procedures because innovative educational 
technologists offer such a wealth of appr cache Sg devices^ systemsg 
etc. g from which to choose for any imaginable instructional task. 
No well -documented rational method now exists for selecting the 
media of instruction best suited for accomplishing a particular 
educational objective ^ 

A strategy is neededg theng which through analysis 
of educational tasks and instructional resources (i. e. g media 
characteristicsls assures optimal matching of educational tasks 
or goals with media instruction. Such a strategy must consider 
several essential elements “ 
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What are the specific educational objectives 



8* o 

to be achieved? 

bo What types of learning activity are involved 

in the educational tasks ^ 

c» What are the conditions surrounding the 



learning situation? 

do For the type of learning and conditions of 
the learning situation identified, what media characteristics are 



required? 



6o What are the characteristics of available 



media systems? 

2o Solution . The most critical element of the 
strategy is determining the relationship between an educational 
goal and the characteristics of media systems. This relationship 
in other words, is that which is found to exist between the ex- 
pected outcome of instruction and the alternate approaches to in- 
structional presentation producing the learning. 

A five step process for determining the media suit” 
able for accomplishing specified educational objectives follows: 

ao STEP ONE - Prepare behavioral objective 
i, e. s state in behavioral terms the objectives for the course 
element or unit of instructiono 



The important components ares 
Action Verbs 
Locus of Action 
Antecedent situation Sg etc. 



b STEP TWO - Describe the types of learn- 
ing which can be inferred from the specified behaviors. Select 
one or more types from the learning cate gone s» 

When selecting learning caiegoriesj tne 
selection is to be made among. 

Verbal Association 
Multiple Di sc r imination 
Perceptual Motor Skill Learning 
Concept Learning 
Principle Learning 
Problem Solving 

c STEP THREE - With these behaviors and 
learning types in mind describe one or more instructional 
strategies which are being or could be used to accomplish the 
objectives. Write the teaching/learning strategy description in 

such a way as to include the following elements; 

Each situation can be described in terms of: 
(1) Mode of Instruction 

(a) Presence 

Live 

Recorded 

(b) Ratio of Personnel 

Gr oup 
Individual 

(2) Sensory Mode 

(a) Re cording /Delivery Symbology 

Iconic 

Digital 



o 
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(b) Receptor Mode 
Visual 
Auditory 
Kinetic 

Feedback to Student 

(a) Immediacy 

Immediate 

Delayed 

(b) Symbology 

Ic onic 
Digital 

(c) Sp ecificity 

General (ioe.g correct/incorrect) 

Corrective (tells what’s wrong or 
right) 

Prescriptive (tells what to do next) 
(4) Provision for Student Response 

(a) Immediacy 

Immediate 

Delayed 

(b) Expression level 

Constructed 
Covert (un emitted ) 

Selected 

do STEP FOUR - Identify the various media 
alternatives which best fit the objectives and instructional 
strategy descriptiona 

One procedure for exploring system com- 
ponent alternatives is to examine the following list of media 



elementSo It is organized into general classes of "Verbal and 
Pictorial, " and is further broken down according to the form of 
the "realness" of the message (i»e„j still vso motion and printed 
vs„ uttered) and into sub -classes based on the physical properties 
of the medium (ioeos paper, film, oxide, etc» )o 

fl) Verbal 

(a) Printed 

(1) Film 

(a) Microforms (i. e: ERIC materials) 

(b) Stripfilm (ioe; MAST device) 

(c) Transparencies (i. e: Vugraph) 

fd) Slides (io e: 2 x 2 and lantern) 

(2) Paper 

(a) Books (the text material in pamphlets, texts, 

et al) 

(b) Worksheets (handouts-charts -forms) 

(c) Panels/Charts (bulletin boards, posters) 

(d) Games ("Battleship" and paper simulations) 

(e) Roll forms (teleprinter, computer printout, 

etCc ) 

(f) Embossed Sheets (Braille forms, punched 

cards, etce) 



(b) Spoken 

(1) Oxide on film 

(a) tape (1/4 in audio tape) 

(b) disks (computer memory) 

(c) belts (dictating devices) 

(d) flat stock (Polyflax device) 

(e) stripe on film ( 8 & 16 mm) 

(2) Radio (broadcasts for educational uses) 

(3) Optical 

(a) on film edge (16 mm & 8 mni optical track) 

(b) in alternate frames (Kalart device) 

(4) Mechanical 

(a) discs (recordings) 

(b) belts (dictaphone, etc* ) 

(5) Realia (live, spoken lectures) 
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(c) Manual 

(1) Semaphore (flags, blinker, etc. ) 

(Z) Hand signs (deaf manual alphabet, etCo ) 

(2) Pictorial 
(a ) Still 

(1) Film 

(a) Filmstrips (35 mm single frame) 

(b) Slides (2x2 and lantern) 

(2) Paper 

(a) Books (the pictorial material in manuals, 

texts, et al) 

(b) Loose Sheets (pictures, maps, other flat 

repre sentations ) 

(c) LDX/FAX (long distance Xerography, 

Facsimile, etc. ) 

(3) CRT 

(a) Conventional TV (used for still transmission). 

(b) CRT still devices (Hughes TONOTRON, 

Westinghouse, etc. ) 

(4) Realia 

(a) Objects (labo instruments, training devices, 

etCo ) 

(b) Kits (construction exercises) 

(a) Motion 

(1) Film 

(a) 8-16-35 & 70 mm Film (conventional reel to 

reel) 

(b) 8 & 16 mm loops (cartridged lessons) 

(2) Tape 

(a) VTR (video-taped lessons & exercises for 
playback) 

(3) TV 

(a) Conventional CCTV and open circuit uses) 

(4) Live 

(a) Demonstrations (by instructors, pre-planned) 

(b) Role-play (simulations, games played-out 

by students and discussed) 



e, STEP FIVE - Select the one medium which 
provides an optimum fit for the characteristics disclosed in 
steps one, two, and three from among the alternatives generated 
in step four. The procedure should result in the preparation of 
a media specification for a particular imit of instruction. Usually 
it will call for one media class but in some cases it will result 
in a mix of media. 

The Media Specification should contain such 
elements as scripts, story boards, input and output test items, 

and a determination of its place in overall curriculum. 

Instructors can use the procedure to describe 
objectives, to identify the types of learning involved, and then 
move on to describe the instructional strategy. Based on this 
combination of instructional design activities, they can then 
determine the types of media which would be most effective. 

3. Procedure utilization. First the instructor 
specifies the behavioral objectives for a unit of instruction. For 
the purpose of selecting a medium, each behavioral objective is 
considered separately. A process for selecting the optimum 
medium (or a mix of media) for a grouped series of behavioral 
objectives (which are considered as one unit of instruction) is de = 
scribed later. 

The instructor next identifies the learning category(s). 
These can be deduced in part from the terminal behaviors which 
he has described. To assist him, examples of educational ob- 
jectives and their accompanying learning types can be provided. 
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The instructor now makes a number of decisions 
concerning the optimum mode of instruction. He specifies a 
teaching /learning strategy in some detail. This includes: 

o Instructional mpdo ; group or individiial and live, 
remote, or recorded 

o Sensory mode : an iconic or digital presentation. 

The sensory mode will be either visual, auditory, kinetic or a 
combination of these 

o Response mode : the instructional strategy will 
include information on the type of response to be elicited, from 
the students 

o Feedback mode; the instructor will also indicate 
the type of feedback to students regarding responses which they 
make during an instructional presentation. 

The instructor has now prepared a Teaching/Learn- 
ing Strategy Description (T/LSD) which identifies the character- 
istics of the instructional strategy which he will follow to ac- 
complish educational objectives he has stated. 

Using the behavioral objective, learning category 
and teaching/learning strategy description, the instructor refers 
to materials which will help him identify a set of media alternatives 
which are appropriate for the support of his instructional strategy. 

For example, if he has identified conceptiial learn- 
ing as being implicit in his preferred mode of instruction, audio 
delivery of digital symbology as the sensory mode, provision for 
immediate constructed response by student, but no requirement 
for feedback, he will find that this combination of elements 



is characteristic of Dial Access w/ workbook; radio programs 
w/workbook; phonograph record w/text; etc. 

After selecting a medium for the first objective, 
the instructor would then repeat the process for other behavioral 
objectiveso By listing the objectives and possible media side-by- 
side the instructor then visualizes a pattern of media usage for 
the entire instructional xin.it. 

He may discover that one type of medium appears 
repeatedly as an alternative for many of the objectives. There- 
fore, he may decide to use a single medium to support the in- 
structional strategy for the entire unit. 

On the other hand, he may find that several media 
are required to support the entire instructional unit. If the latter 
is the case, the instructor will strike the best compromise be- 
tween the most logical sequence of presentation of course material 
in the imit and the most practical sequence of media utilization, 
(See Briggs reference, Chapter III). 



I - HISTORICAL DEVELOPMENT 



D , Evaluation of Multi-Media System s 

1. Examination of the variables . The evaluation of 
a proposed or presently operating media system is a key phase of 
the media system planning process* This evaluation requires a 
means for systematically and objectively collecting data about the 
media system. To meet this requirement, AIR expended con- 
siderable effort in the development of a data collection instrument, 
the final version of which, could be administered, summarized and 
interpreted in the field, on-site by an institution performing a 
self-evaluation. 

The first step in developing such an instrument was 
to specify those variables which are significant to the evaluation 
of media sysjtems, A list of the variables identified appears 
below. 

VARIABLES LIST 
a , Identification 

(1) Institutional identity 

(a) Clientele 

(1) Age 

(2) Family income 

(3) Percent commuters 

(4) College Entrance Examination Board 
cutoff score 

(5) Cultural factors 

(6) Religious orientation 

(b) College 

(1) Generalized goals or type 

(2) Location 



(3) Size in student numbers 

(4) Organizational structure 

(5) Alumni influences 

(c) Faculty 

(1) Number 

(2) Percent research 

(3) Percent teaching 

(4) Academic percentages 

(5) Salary levels 

(d) Facility 

(1) Number of buildings 

(2) Size of buildings 

(3) Total square footage 

(4) Scattered vs, compact 

(5) Age of plant 

(6) C ost of maintenance as percentage 
of gross plant value 

(2) Identify system 

(a) Originator 

(1) Title 

(2) Goals 

(3) Funding sources 

(b) Plans 

(1) Original design 

(2) Development stages 

(c) Implementation 

(1) Start-up headaches 

(2) Operational problems 
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Performance 



(1) Student variables 

(a) Numbers 

(1) Numbers into the system 

(2) Number of drop-outs 

(3) Number out of the system 

(4) Percentage attendance or use 

(5) Use voluntary or compulsory 

(b) Personal 

(1) Age 

(2) Sex 

(3) IQ 

(4) Prerequisites for this sample 

(5) Interest measures 

(c) Throughput 

(1) Input testing of baseline knowledge or 
skill 

(2) Output testing of baseline knowledge 
or skill 

(3) Past numbers of output students 

(4) Present numbers of output students 

(d) Academic 

(1) Achievement measures 

(2) Attitude clua-nge measures 

(3) Is system use high? 

(4) Is system use increasing? 

(e) Elective 

(1) Number selecting course 

(2) Number taking follow-on courses 

(3) Career selection information 



(4) Career satisfaction information 

(f) Temporal 

(1) Percent of time saving over con- 
ventiontil instruction 

(2) Or use more time with system in- 
struction? 

(3) How assign value to student hours? 

(2) Staff variables 

(a) Definition 

(1) Have task analyses been written for 
staff jobs? 

(2) Do descriptions exist? 

(3) Are performance evaluations carried 
out? 

(b) Professional staff 

(1) How many persons give how many 
hour s ? 

(2) Are these book hours vs. actual hours? 

(3) Is training provided (type, length, 
locale, and curriculum)? 

(c) Technical staff 

(1) Number of persons apd hours of duty 

(2) Are these hours spent monitoring, 
really working, (i.e., intensity of 
involve m.ent) ? 

(3) What level of competence is expected? 

(4) Is training provided (amount, cost, 
curriciilum)? 

(d) Student assistants 

(1) Number of clock hours of paid student 
assistance 

(2) Number of clock hours of volunteered 
student assistance 
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(3) Required levels of activity for student 
assistants 

(4) Types and expense of training pro- 
vided to student assistants 

(3) Equipment Variables 

(a) Descriptors 

(1) Written description of system 

(2) Use checklist of input and response 
options to specify media mix 

(3) Obtain copy of specifications for 
System 

(4) Product literature on components 
of system 

(5) Special hook-ups or relationships 
unique at this setting 

(b) Fidelity 

(1) Lines of raster for television 

(2) Frequency response characteristics 
of audio portion 

(3) Resolving power of image projection 
system 

(4) Legibility and type size of paper inputs 

(c) Reliability 

(1) Down-time 

(2) Materials damage 

(3) Percent frequency failure during use 

(4) Percent class time lost for all re- 
liability reasons 

(d) Cost/life 

(1) Original costs 

(2) Maintenance costs 

(3) Replacement costs 
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(4) Expected life of equipment 

(5) Actual in-use life of equipment 

(6) Computed cost per unit of equipment 
life 

(4) Teaching material variables 

(a) Sources 

(1) Purchased 

(a) Suitability 

(b) Percent use of purchased infor- 
mation 

(2) In- house developed 

(a) Percent use of home-made 

(b) Cost of development {include all 
staff time and materials) 

(c) Life expectancy of home -developed 

(b) Cost /life 

(1) Accumulate cost and life information 
on off-shelf and in-house materials 
to determine cost per unit of student 
use 

(c) Quality factors 

(1) Ratings of independent judges 

(2) Ratings of students 

(3) Fidelity level of input materials 
(equipment variables, see ^Fidelity" 

above) 

2. An experimental, respondent- oriented evaluation 
instrument; FIELD . Questions were constructed to gather infor- 
mation about these variables. These questions were then struc- 
tured to form the data collection instrument which was named 
FIELD (Field Instrument for Evaluation of Learning Devices). 

More about the construction and use of FIELD follows. 
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S'* Data collection e Information was collected 
along two general dimensions: predictor variables and criterion 

variables. Predictor variables are those many factors and ele- 
ments of the teaching /learning system design and operation which 
might be expected to have bearing on the effectiveness of the 
system. Criteria variables are the few factors which serve as 
output measures or comparisons of students* accomplishment 
with collegiate goals. The variable classes consisted of the 
variables listed earlier categorized under these two areas. 

Both types of variables were ftirther broken 
down into other general clusters: 

o STUDENT Variables 
o STAFF Variables 
o EQUIPMENT Variables 
o TEACHING MATERIAL Variables 
o FACILITY Variables. 

b. Data sources. Various data sources were 
tapped by on-site visitation teams. The four major avenues for 
application of the tool were: 

o On-site inspection of the multi-media 
system in operation 

o A study of the original plans and other 
documents associated with the development 
and use of the system 

o Depth interviews with the technical and pro- 
fessional staff of the system- 
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o Interviews with students both in process 
and those who had finished using the system 
or dropped out of it 

c , Selection of participating MM systems for 
inclusion in the pilot study . It was proposed that at least one 
system from each of the categories listed above (print- structured, 
audio-linear, local multi-media, electronic distribution, and 
student activity) be selected for intensive study during the pilot 
trial runs of the FIELD. Language laboratories have achieved 
wide acceptance,, Each institution of higher education selected 
for study had a langxiage laboratory, in addition to the specific 
system under study, which was also analyzed with FIELD. 

d. Activity schedule . Since an evaluation 
team had to go into the field and conduct the study, it seemed 
important to select institutions which were receptive to the con- 
cept of the FIELD development. Therefore, to select 

such a group of institutions, the activity schedule outlined below 
was followed: - , 

(1) Contacted a selected group of educators 
(with specialized skills and interests). 

(2) Requested that they serve as members 
of a panel to review the first draft (FIELD) in early 
December, 1967., 

(3) Invited this panel of experts to meet with 
AIR staff during a training period in early January. 

(4) Met with experts to plan the administration 
and interpretation of FIELD, 
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(5) Asked panel members to * Volunteer” the 
media system at their home institution as a guinea pig for the 
first round of elevations. (Thus, the panelists would have an 
opportxmity to influence^ if not control, the structure of the evalu- 
ation instrument and to participate in tne briefing of AIR staff 
prior to the administration of FIELD). 

(6) After development of a final instrument 
incorporating the best understandings generated by AIR staff 
and supplemented by panel members’ contributions, assigned 
AIR staff to two-man teams to take FIELD to the pilot institutions 
for experimental administration , 

(7) Analyzed results of pilot administrations, 
prepared FIELD study findings and specifications for NEW 
FIELD. 

e. Timetable . Activity in late October and 
early November was devoted to the design of a preliminary in- 
strument and to the selection of a panel of experts. 

During November and early December the 
panel, (having volunteered the use of their multi -media systems 
as vehicles for the FIELD development) was employed by AIR as 
consultants to assist in FIELD de -bugging and development. In 
late December, a second generation version of FIELD was com- 
pleted and a two or three day meeting was held in January at a 
central location. 

This meeting was attended by the panelist - 
consultants and AIR staff. Based on the further de -bugging and 
training activities which took place at that time, a final instru- 
ment was developed at AIR, 
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This was applied by field evaluation teams 
of two persons each at the volunteer institutions. AIR staff worked 
with panelists to arrive at an appropriate FIELD reporting system. 



II FIELD STUDY 

A. Methodology 

1. Purpose . The purpose of the Field Instrument for 
the Evaluation of Learning Devices (FIELD) is to produce useful 
information concerning the cost, utilization, pattern, and overall 
effectiveness of existing and future multi-media systems in higher 
educsttion. 

2. Instrument design; variables. The first draft of 
FIELD was built upon the list of predictor variables and criteria 
variables described earlier. 

From the list, a series of individual questions designed 
to elicit data concerning each variable was prepared. They were 
organized into three booklets. One booklet asked questions identifying 
the institution. The other two asked questions on the personnel 
(students and staff) and equipment and software varia-bles. 

These questions were prepared in an item -by -item 
format with space for panelists to indicate: 

o How best to obtain the material (respondent category) 

and 

o What format to use in obtaining the information 

(instrument category). 

Upon completion of the full set of question^ this draft 
material was mailed to participating media specialists. 

3. Panelist selection . Panelists were selected for 
their special knowledge and interest in specific classes of systems. 
Each panelist came from a university or college where two or 
more multi-media teaching systems were in use. 
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Once the list of media specialists and systems was 
established, panelists were contacted and requested to serve as 
consultants to review the first draft of the evaluation instrument. 
Each was asked to volunteer the media system at his home in- 
stitution as a site for a tryout of the FIELD. This gave the 
panelists an opportunity to influence the content and structure of 
the evaluation instrument. Panelists participated in the briefing 
of AIR staff prior to the visits to their campuses for the administra 
tion of the FIELD. 

The media specialists, universities and media systems 
represented were: 

Panelist and Institution 

Lawrence Stolurow 
(harvard U. ) 

John Childs 
(Wayne State U. ) 

Kenneth Fishell 
(Syracuse U. ) 

Harvey Meyer 
(Florida Atlantic U. ) 

4. Revision activities. The first working draft of the 
FIELD was completed and sent to the consultants in the form of 
three pamphlets: 

(a) Performance, Equipment, Materials 

(b) Performance, Student, Staff 

(c) Identification, Institution and System 

Panelists were asked to assign respondent and instru- 
ment categories as indicated below: 
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System class 



Specific systems 



Printed Structured 
Audio- Linear 

Audio -Linear 
Audio - Linear 
Active 

Local MM 
Audio - Linear 

Distributed MM 
Audio - Line a r 



CAI 

Language Laboratory 
DA IRS 

Language Laboratory 
VTR (micro-teaching 

Multi-Media Classroom 
Language Laboratory 

CCTV 

Language Laboratory 
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Respondent Categories 
Student 
Technician 
Professor 
Other 



][nstrument Categories 
Questionnaire 
Interview Schedule 
Othe r 



Along with assigning means and sources according to 
respondent and instrument categories, panelists were asked to 
weed out those questions which were inappropriate, amend others, 
and include additional questions as required. 

5. Preparation of interim draft. As the evaluations 
were returned, a mrthod for restructuring the FIELD questions 
for consideration by the panel was established. The respondent 
categories were expanded and used in conjunction with such factors 
as cost, time, affect measures, goals and effect, and system 
description. 

A chart is attached indicating the pigeon-holei- into 
which questions were sorted for the interim draft, according to 
respondent and type of information called for. The resulting sets 
of questions were numbered and considered in batches during a 
meeting of panelists convened for that purpose. The second 
draft was finished and the panel meeting was held 11 and 12 
January 1968. 



The media specialist- consultants attended together 
with AIR staff members Dr. Harold P. Van Cott, Dr. George 
Johnson, Dr. Ronald Carver, Mr. Charles Williams, Mr. 
Christopher Faegre, Mr. .3ohn Connolly and Dr. Ray Muller. 

Other panel members were; Dr. Tongsoo Song, Mr. Peter Esseff, 
and Mr. Alfred Dubbe from the U. S. Office of Education. The panel- 
ists discussed the contract objectives and the short-term conference 
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CHART I - INTERIM FIELD 



RESPONDENT CATEGORY 
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goals. After this, small groups were formed and the questions 
(classified according to information classes and respondent 
categories) were discussed. At this same meeting arrangements 
for on-site visitation dates and self-selection of interviewing 
teams took place. Panelists had an opportunity to meet interview- 
ing team members. An overall understanding of the complexity 
of the evaluation task was gained. The diversity of opinion regard- 
ing wording of questions and appropriateness of questions was help- 
ful in developing an instrument which would be readily understood. 

The attempt to classify questions by information class 
and respondent category was, however, frustrating. Qu«*stions 

which made good sense in their first location in the variables list, 
suffered by being lifted out of context and shuffled together with 
other questions from other aspects of the original variables list. 
This made the task of group review and revision of questions dif- 
ficult, Nevertheless, an appreciation of the breadth of the evalua- 
tion task did emerge from the conference, and it was partly be- 
cause of the organization of questions for review that this sense 
of complexity and importance was evident. 

6, FIELD tryout version. Using the notes and sug- 
gestions of panel members, the FIELD was reworked into a 
"tryout" form. It was now ready to be carried to. the various 
university campuses by the AIR teams. 

The refined FIELD was broken down into seven respon- 
dent categories as follows: 

o STUDENT (User of MM System) 

© STUDENT (At Large) 

© TEACHER (User of MM System) 
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FACULTY (At Large) 

TECHNICIAN 
o SYSTEMS DIRECTOR 
o INSTITUTIONAL RESEARCHER 

The questions in each, respondent category were then 
arranged on the page with introductory statements defining the 
reasoning or information need which was being served by the parti- 
cular question. This aided the system evaluator to interpret ques- 
tions and demonstrate utility in evaluating the effectiveness of the 
particular system. 

A copy of the FIELD, as s?evised in the FIELD tryout, is 
included as Appendix A. 

7. FIELD application. 

a. Visitation strategy. After the systems and the 
institutions to be studied were selected and the FIELD had been pre- 
pared, the actual on-site visitations began. 

The following Table indicated the institutions visited 
and the faculty members contacted: 



Institution and 


Systems 


AIR staff 


Faculty Contact 


Studied 


interviewers 


Florida Atlantic 


CCTV 


Dr. H. P. Van Cott 


Meyer 


Language Lab. 


Mr. Charles Williams 


Syracuse University 


Multi- Media 


Dr. George Johnson 


Fishell 


Language Lab. 


Mr. John Connolly 


Harvard University 
Stolurow 


CAI Laboratory 
Language Lab. 


Dr. Ronald Carver 

Mr. C. Faegre 

Mr. Charles Williams 


Wayne State University 
Childs 


Micro- teaching 


Mr. Charles Williams 
Mr. Charles Darby 


Wayne State University 


Language Lab. 


Mr. C. Faegre 


Childs 


Dial-Access 


Dr. Raymond Muller 
Mr. Charles Darby 
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During the visitation schedule, an orientation, training 
and familiarization session enabled AIR staff members to have 
mock interviews with Dr. Van Cott and Mr. Williams, both of 
whom had earlier conducted interviews at Florida Atlantic Univer- 
s ity . 

As further visits were conducted, additional efforts 
were made to familiarize AIR staff members with team experiences, 
so that orientation and training were continuous throughout the 
period of administration. Members of the teams shared their con- 
cerns about the document and instruments and kept a running modi- 
fication of the instrument in mind as they conducted the interviews. 

At each institution the AIR staff personnel met their 
university contact and received a brief tour of the system facility 
that they were to study. They then requested that interviews be 
arranged with faculty members using the system as well as with 
some who were not using the system. Interviews were set up with 
such system personnel as the Director and his technicians. Ar- 
rangements were also made for group administration of the student 
questionnaire to classes of students who had used the system. 

Some student non-users were interviewed individually 
by approaching them in classroom buildings and asking for an in- 
terview regarding their experience with media systems. 

b« Time required for administration. The follow- 
ing is a Table of questionnaires which were administered. They 
were titled according to the respondent being interviewed. The 
average length of time in minutes required to administer the 
questionnaire is given in the table: 



Time in • 

Questionnaire Minutes 

Student User 25 

Student Non- User 15 

Teacher User 60 

Teacher Non-User 15 

System Technician 120 

System Director 180 

Institutional Researcher 60 



c. Administration . Application of the FIELD at 
the universities followed a fairly consistent pattern. For most 
interviews two AIR staff members worked with one respondent. 
One AIR staff member was responsible for conducting the inter- 
view and recording the responses of the interviewee on the ques- 
tionnaire forms. The second AIR staff member observed the 
interaction between interviewer and interviewee. Questions 
which caused difficulty were noted. Ideas for rexdsion of 
questions were entered into a Master copy of the FIELD. 

Where additional explanations were required they were noted. 
Other riinning commentary and impressions were recorded by 
the observer. 

d. Departures . Several departures from the 
original plan for administering the FIELD were taken. For 
example, since the time consumed in administering the FIELD 
was greater than anticipated, most Student Users were asked to 
respond to the FIELD in classroom groups rather than as 
individuals. 

The particularly long questionnaires (such as the 
Teacher Use?:, System Technician and Systems Director) were 
usually conducted in two or more sittings, rather than all at one 
sitting. In some cases, it was necessary to administer the same 
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questionnaire in parts to two individuals in the same respondent 
category. For example, the first half of one technician question- 
naire was administered during one interview session. The second 
half of the same technician questionnaire was administered to a 
different technician involved with the same system. In other 
situations where there was only a short time to finish a question- 
naire, some items, about which a great deal of information had 
been previously gathered, were skipped. This allowed additional 
responses to vital questions to be secured. 



II. FIELD STUDY 



B. F inding s 

1= Findings concerning the instrument. Based on the 
findings from the FIELD, specifications have been developed for a 
revised FIELD which will be presented and discussed in Sec = 
tion II, D of this final report. 

a. Definition. All AIR personnel who administered 
the FIELD mentioned the problem of definition. Examples of problem 
terms are: "media, " "system, " "educational activities, " "functional 
specifications, " and "technical specifications, " among others. 
Another aspect of the problem related to context, in which the 
responding individual must place himself to answer the questions„ 

For example, it was not always clear in the questionnaire whether 
the individual respondent should base his answer on his total college 
experience in answering the questions or only on his experience 
with mediated courses. If he were to answer questions in terms 
of mediated courses, should he report this experience with a3.1 
media systems or just the one under study? Another very crxuical 
definition of this type related to the meaning of "system user. " 

In some cases, students who were users of the system were asked 
to answer non- user questionnaire because the initial series 
of questions did not determine whether the individual was a "system 
user. " In addition, several non-users had had previous experience 
with the media system but at the time of the questionnaire admini- 
stration were not using it or had no experience with the particular 
media system being analyzed. 

Two other definitional problems are; the definition of system 
or class under which the subject system falls, and of the particular 
system as provided in a functional relationship diagram. The 



FIELD questionnaire provides a panel which shows a diagram of 
the systems under study. In some instances, individuals were 
unable to determine whether the particular system with which 
they had experience was represented. 

System personnel were asked to sketch the functional inter™ 
action of their media system. The sketches ranged from repre- 
sentation of the interaction between hardware and the interaction 
between course material and the student. 

b. System specificity. Prior to entering the trial 
phase, all questions in the FIELD were conceived to be applicable 
to all types of systems. Administration of the FIELD proved this 
to be a difficult (although not unworkable) concept. Numerous 
questions apply only to specific types of systems or must be asked 
in a system- specific form. Later editions of the FIELD should 
include both questions which are generally applicable to all systems 
and some sets of questions which are geared to specific systems. 

c. Teacher inclusion . As indicated, respondents 
were requested to draw a diagram of the media system. In some 
of these the teacher was shown as part of the media system, while 
in others the teacher was not shown. This variability may have 
caused a difference in students’ responses regarding the system, 
i. e. , if students considered the teacher as part of the system, 
their reactions were to the teacher rather than to the media itself. 
However, it is realistic to consider the teacher as part of the 
system, since media systems seldom function in isolation without 
"course framework" or assignments, etc. The teacher functions 
as an integral part of the system using the media to accomplish his 
instructional goals. 
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d. Duplication . A number of questions in the 
FIELD questionnaire have been duplicated by design. The same 
question was asked of respondents in two different categories as 
a means of obtaining comparable data on two populations. For 
example, the same question regarding the effectiveness and 
efficiency of the hardware is asked of both the Systems 
Director and faculty members using the system. It is obviously 
important to determine whether system personnel and system users 
share the same view regarding effectiveness. These types of 
questions attempt to measure the effectiveness of media systems by 
sampling attitudes of those involved with them rather than by 
collecting factual information. These attitudes must be verified by 
sampling the attitudes of respondents who potentially may hold 
opposing views about media systems. 

e. Attendance constraints . Faculty users were 
asked to indicate whether the use of the media system is voluntary 
or compulsory for students. Responses to this question indicate 
that attendance constraints cannot be dichotomized in an all-or- 
nothing manner. In most cases, attendance is neither voluntary 
nor compulsory, but in fact, falls on a continuum between those 
two extremes. In any one course, students' use of the media for 
some portions of the course may be voluntary and for other por- 
tions of a course may be compulsory. In another situation, the 
instructor does not require his students to visit the laboratory and 
does not take attendance, yet does present material through the 
media system which is not covered in his lecture ard which he 
expects the student to learn in order to complete course requirements. 
Such a situation may cause students to perceive attendance as 
compulsory while the instructor considers it to be voluntary. 
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Some students may be so concerned that they will miss part of the 
course material that they consider the media laboratory to be 
compulsory, at least for themselves. 

One other factor clouds the issue of attendance. When media 
are used as part of a classroom presentation, attendance constraints 
are those placed on classroom attendance. An accurate measure- 
ment of the degree to which attendance is compulsory or voluntary 
must be obtained because the interpretation of the data relative to 
system utilization is obviously quite different under the two conditions. 

f. Choice. Students were asked in the FIELD 
questionnaire whether they have the option to choose media 
taught courses over conventionally taught courses. Experience 
indicated that many students had no way of predicting whether a 
course was taught using media or conventional means. The 
catalogs of classes which students used to select their courses 
gave no such indication. Questions concerning the students 
selection of media vs. conventionally taught courses may have 
to be asked in terms of a hypothetical situation only. 

§• Deveiopnient vs. operationa l. Systems studied 

/ 

during the FIELD trials tended to fall in one of two categories. 

Some went through developmental stages and later became operational; 
others remained experimental and were used primarily for research. 
The existence of these variations created additional difficulties in 
evaluation of the FIELD. 

h. Costs. All cost information was very difficult 
to obtain. Most questionnaire respondents had not performed a 
careful cost- analysis on the media system for which they were 
responsible. Those who ha4 did not approach the problem in the 
same manner as did the FIELD. Cost information is essential to 
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evaluation of system effectiveness. The information to be gathered 
must be carefully selected on the basis of its significance for 
measuring cost effectiveness. 

Question format . Many respondents had diffi- 
culty in answering forced- choice questions, i. e, , those which 
give only "Yes-No" choices or some other limited number of 
choices. 

j. Systems app r oac h. While the FIELD asked 
questions concerning media system development as though colle- 
giat e level planning were a systematic and orderly activity, none 
of the respondents at the institutions visited had experienced 
opportunities to systematically develop their media systems. 

k. Measuring system effectiveness. Several re- 
spondents to the FIELD questionnaire were concerned with 
measuring system effectiveness. They felt that the present form 
of the FIELD did not investigate the issue sufficiently. One re- 
spondent suggested that questions be asked to determine if use 

of media tends to promote or stifle favorable attitudes toward 
the subject being taught. 

Another respondent suggested that system personnel and 
faculty users be permitted to indicate present and future ways in 
which to m-easure system effectiveness. Others mentioned the 
importance of determining, through the questionnaire, whether 
student learning is improved through the use of mediated instruc- 
tion. 

l. New areas for study. Other areas suggested by 
respondents involved specifying the kinds of learning each type of 

media system is used for in practice and with what degree of 
success. (See Part I, Sec. C. ) 



Several System Directors and faculty members indicated that 
an important factor in planning and developing a system is the 
amount of administrative support received. Related to this is the 
issue of funds. One individual respondent felt that it was necessary 
to distinguish between size and criticality of funds. He indicated 
that the most common problem was a lack of follow-on funding for 
staff and software development. He noted that it is com- 
paratively easy to obtain funds for system hardware but difficult to 
obtain funds for staff. It was also mentioned that the FIELD 
questionnaire might be adapted to a case study approach or profile 
describing how an individual faculty member decided to instruct 
using a media system. 

m. Acceptance of FIELD. Most interviewees felt 
that the probing questions asked by the FIELD should stimulate 
thinking about media system evaluation. 

^ • Findings concerning the institutions and media systems. 
Many of the issues discussed in this section have been commented 
upon in the previous section in terms of the FIELD instrument. 

The following discussion centers on the influence of these issues 
on the institutions aind media systems studied. 

a. Teacher inclusion. The influence of the instruc- 
tor on the effect of the media system is one such issue. The 
teacher, in most instances, is considered to be part of the system, 
since he normally produces some or all of the curriculum materials 
for the system and sometimes participates as a live user of the 
media system. The attitude of students toward an instructor may 
have great influence on their attitudes towards the media system 
of which the teacher is a part. 
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In some systems, the teacher is a component of the system 
during its actual operation. Including the teacher as part of the 
system to be evaluated adds a new dimension to both the capa- 
bilities of the system and the problems faced when designing it. 
Therefore, media system designers should take into account the 
capabilities of the faculty who are potential users and participants 
in the media system. 

To specify the interaction of equipment in a media system 
may be easy, but specifying effective interaction of teacher, 
equipment, and student in course material presentation is far 
more difficult. 

b^ Developm it vs. operational stages . When 
educational decision-makers were evaluating or planning media 
systems, they considered the developmental stages prior to 
the operational stages. Even during the operational stage 
development of increased equipment capacity and software library 
continued. Because the nature of media systems changes as they 
pass through the stages of development to become mature opera- 
tional components of the higher educational armamentarium, such 
systems have to be evaluated differently. A system in an early 
stage of development cannot be expected to be cost effective, 
whereas a fairly mature system may be measured by a rigid 
cost accounting paradigm. 

In addition, when planning a system, the designer must 
realize the differing factors of cost during the various stages. 

For example, hardware costs may be the most expensive at first; while 
later, software development costs may predominate. Also, a 
system may move from an early isolated configuration into a 
more sophisticated system interconnected with other media 
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systems on the individual university campus or other systems at 
other educational institutions. 

Co T ime - s aving s „ Users of media systems 
generally indicated that they did not save time through usage. 
Preparation of materials took much longer than for conventional 
classroom lecture presentation. 

However, most system users felt that after course materials 
had been fully developed and packaged for presentation, time was 
being saved. Therefore, in the early use of a media system, sys- 
tem planners should be prepared for higher faculty c osts and, less 
e fficient use of manpower. Over the long run, the opposite pre- 
vails. This may not be true of media systems which require 
continuous additional time on the part of instructors, but a resultant 
increase in quality of instruction will offset the time increases. 

d. Faculty compensation. Respondents to the 
FIELD questionnaire repeatedly emphasized the importance of 
developing good software for a media system. At the same time, 
software is the most difficult commodity to obtain. In most 
institutions, faculty memibers are responsible for software development. 
Since increased time is required for this, some form of released 

time or use of special faculty staff members is recommended to en- 
courage participation in software preparation^, 

Although released time was the most frequently mentioned 
compensation, other benefits might include monetary remuneration or 
special positions, privileges and promotion for those faculty 
members developing software for use in media systems. 

e. Copyright i^sue. Several faxulty members 
also voiced concern regarding their copyright to software and course 
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materials developed for use in media systems. Do the materials 
which have been developed in fact belong to the faculty members 
or to the university? In addition, several faculty users felt that 
if the materials an instructor had developed were used at another 
institution, he should be paid royalties for the materials. 

A distinction must be made between state and privately 
controlled and supported institutions of higher education. It is 
most common for the faculty member of the first type to give up 
all rights to materials he had developed, unless he can demonstrate 
that the materials were prepared on his own time. In private 
institutions, this rule is not so rigidly observed. 

In some cases, instructors in state supported schools feared 
that completion of a set of software materials for a course would 
have the unwanted effect of getting a second course assigned to them 
once they had the first course "in the can. " They feel they might 
"work themselves out of a job." 

Several persons operating media programs expressed con- 
cern about rights to materials and the effect which conflicting rules 
concerning copyrights can have on faculty creativity. System 
designers should build software development incentives into the 
program rather than fight the apathy created by the situations in 
which no right or interest resides in the writer or developer of software. 

f. Systems approach and behavioral objectives. 

The FIELD questionnaire attempted to study the effect of a process 
(System Planning) which is usually assumed to be a systematic one. 

In most cases, the system was not planned systematically. 

As indicated, the FIELD may serve to stimulate those responsi- 
ble for designing media systems to take such an approach. Nearly 
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all respondents recognized the need for systematic develop- 
ment but were also painfully aware of the difficulty involved. 

One important aspect of the development of a media system is 
the defining of the output from the media system, that is, the 
learning that will take place as a result of the use of the media 
system. It is at once the most important area of planning the 
most neglected area of planning and the most difficult activity 
to accomplish. 

Faculty users and system personnel frequently averred 
that "educators know very little about the outcomes expected from 

teaching. " They are not sure what behavior they want students to 

/ 

display a.s a result of teaching. To glibly say that one should 
identify the outcomes of a course in behavioral terms grossly 
underestimates the problems involved. A considerable length of 
time and large amounts of money to empirically define those 
behaviors which students should display at the end of a presenta- 
tion of a course will be required. 

g. Funding . In tracing the history of media 
systems, it frequently appeared that the unexpected or unearned 
availability of funds was a prime mover in initiating action for the 
development of the system. Only rarely was an orderly procedure 
followed in the development. 

h. Selection vs. evaluation. There was some 
evidence to indicate that the decision making process involved in 
selecting a new media system from a group of alternatives may 
be different from that involved in evaluating an existing system. 
For this readon, FIELD data may require differential interpre- 
tation. 



i. F amiliarizat ioii o The job of the educational 



decision maker and system planner seemed to include one of 
publicizing existence and the potential of the media system on the 
university campus. Nearly all faculty members and system per- 
sonnel pointed to increased familiarity with a system as the best 
way to increase its use. A1 so, nearly all faculty members who 
indicated they were not using the media system did not know of its 
existence. To receive the ultimate benefits from the media system, 
system planners must be prepared to proselytize and train the 
faculty in the use of the system. 

j. Administration . The system planner must 
also take into account requirements for administration. In some 
instances, non- academic personnel administer the system, in 
others, academic personnel were responsible. It seemed that 
there may be an optimum mix of both academic and non- academic 
personnel for planning and operating a media system. In any 
event, sufficient administrative support and back-up is required, 
particularly in the early development to assure full support by 
those who are most influential in the area of staffing and funding. 

k. Achievement. Although the education 
decision maker may gather a wealth of evidence to indicate that 
both students and faculty members like a media system and tend 
to use it, officials responsible for funding such systems may still 
require evidence that the media system does in faict promote more 
(and possibly more rapid) learning than conventional lecture 
methods . 

Most institutions of higher education resist the idea of 
evaluating instruction. It is far more common to evaluate student 
achievement, placing the responsibility for educational progress 
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on him. Efforts to measure instructional effectiveness are resisted 
because they threaten to put the onus for low achievement on instruc 
tionai personnel and methodologies. 

This single problem (the finding of adequate measures of 
effect) was probably the most difficult aspect of the FIELD 
development and try-out. Only one institution (Wayne State 
University) was running a significant number of students (approxi- 
mately 200) in a discrete system (a political science course 
being taught with the partial use of a Dial access lab) which had 
comparable non-mediated instruction going on at the same time 
with similar groups (conventional political science instruction 
through lecture and discussion). And even in this situation, there 
were inadequacies in the pre-test and post-test materials available 
for use with both groups. 

It may be, as is suggested later, that the present state-of- 
the-art will require use of "credit hours" and "grade points" as 
rudimentary measures of achievement and. output effects. 

3. Nu merical data findings. Data related to 
certain key issues such as student use, faculty use, software cost, 
hardware cost, media vs. conventional instruction preferences, 
etc., will be reported in this section. These data, although tabu- 
lated, have not been prepared in final form. Data for this section 
are based on the FIELD administration at four institutions of 
higher education and represent nine media systems. 

Collection of data on nine media systems in four universities 
was designed as a trial of the FIELD. It was not intended to gather 
evaluative data so much as to test the feasibility of gathering data 
about media systems. As a by-product of testing the FIELD 



instrument at four institutions, a great deal of data were collected. 
Significant samples of this material are reported in this section. 
The data aggregations reported here are based on only a small 
number of observations. They in no way represent a sample of 
observations of media systems in higher education institutions 
upon which generalizations can be drawn. 

These data, however, were carefully and objectively 
drawn from the four institutions visited during the FIELD tryout. 
Therefore,, they do provide some information about media systems 
at those institutions. Information reported is that which was 
particularly significant or interesting and about which data could 
easily be tabulated from the trial FIELD. Findings based on 
this field data are discussed in the following paragraphs. 



a. Demand . To measure effectiveness, it is 
necessary to determine the degree of utilization. Students were 
asked to estimate the number of hours beyond the scheduled 
class hours of system use they used the media system. The 
question was phrased: "How much time do you spend using 

media beyond the scheduled class hours per week? " The phrasing 
was unfortunate in that it requested estimates of extra- class 
utilization only. Future questions will request estimates of 
both in class and out of class use. 



Of the 56 students who responded to this question, 26 
indicated they they did not use the media beyond the classroom 
at all. The median use beyond scheduled classroom time was 
45 minutes/week and the mean use of the system was just in 
excess of one hour /week. Students' responses ranged from 0 
hours of use to a high of 6 hours of use on a weekly basis. 
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b. Time saving: student s. Another measure of 
the effectiveness of a media system from the standpoint of a 
student is whether time is saved. "Do you feel that time is 
saved when using an instructional system (compared to 
conventional methods of instruction)? " was asked of the students. 
Comparison was made to conventional methods of instruction in 
order to give the student a frame of reference on which to base 
his estimate. 

Of the students answering this question, 76. 8 percent 
felt they had saved time. 

c. C ourse selection . The respondent was asked, 

"If the same course were being offered two ways (one 

using conventional methods and one using mediated methods) 
which would you choose, all other things being equal? " Students 
indicated that actually they had no knowledge of the instructional 
mode of a course as described in the schedule of classes so that 
this question and the one that follows were asked about hypothetical 
situations, 

A high percentage (81. 4%) indicated that they would choose 
mediated instruction over conventional instruction. 

d. Preference. The next question was designed 
to determine if the student would choose a course knowing that 
it was taught using a mediated system approach. 

Students were asked, "Would you choose another course 
taught by the mediated instructional system approach? " 

A larger percentage (86%) chose mediated instruction in 
this case than in the previous question. The slightly larger 
percentage of student answering favorably toward mediated 
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instruction is worthy of comment. It may be that there are students 
who would not reject a course simply because it employed a 
mediated instructional system. However, these same students 
would reject the media approach if they could take the sapie 
course using a conventional instruction system. In any event, 
there was a favorable attitude toward mediated instruction. 

e. Student los^dir" Faculty members were asked 
to report on their u^.© the media system under study. One of 
the first qu^esttons asked them dealt with the student load they 
wejre liandling with the media system., Faculty members were 
asked three questions leading up to an estimate of the gross 
number of student hours per week that the media system was 
being used by the individual faculty member in his courses. 
Questions were as follows: 

(1) How many students do you have 
participating in courses using this teaching media system? 

(2) How many hours per week (on the 
average) do they use the system? 

(3) What is the gross number of students 
per week calculated from above? 

Estimates of the gross number of student hours per week 
ranged from a low of 12 student hours per week with one language 
laboratory to a high of 6,175 student hours per week with a 
computer aided instruction system. This tremendous variability 
in the small number of observations reported upon makes 
summary statistics rathfir questionable. Therefore, only the 
actual observations are shown in the following chart. 
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CHART J - INSTRUCTORS' CALCULATI ONS OF MEDIA SYSTEM USE 



Gross No. of 

Medi a Sys tem Student Hours /Week 



ETV 


300 


CAI 


6, 175 


DAIRS 


35 


dairs 


112 


DAIRS 


360 


DAIRS 


42 


ETV 


400 


MM 


180 


MM 


375 


MM 


42 


LL 


12 


LL. 


1, 000 


LL 


250 


LL 


30 


LL 


323 
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f. Time saving: faculty . Student generally viewed 
the use of media systems as a time- saver. Faculty members 
were asked their views on this subject. 

The question was phrased: "With respect to time-use 

when teaching with mediated instructional systems, do you find 
that you use more time overall or do you save time overall? 

Unlike students, faculty members indicated they they used 
more time in courses where they were utilizing mediated 
instruction than in those in which they were utilizing conventional 
instruction. Of the faculty members responding, 87.5 percent 
indicated they they used more time with mediated instruction 
than with conventional instruction. 

They reported that a good deal of time was spent preparing 
software materials for their media taught courses. Once the 
software was developed, the majority of them believed that 
time would be saved. The additional time required to prepare 
software was taken from the faculty members’ own free time 
rather than time planned for other job activities. 

g. Training: faculty . One essential element of a 
successful media system is the training of participants. To 
determine whether faculty members were satisfied with the 
training which they had received, they were asked this question: 
"Do you feel your training has been adequate to effectively 
handle this instructional system? " Of those responding, only 
40 percent felt the training had been adequate. Training 
received by participating faculty members had not been of a 
formal nature. Rather, it consisted of assistance from system 
personnel, faculty members, or self acquired knowledge. 
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h. Faculty satisfaction. Still another measure of 
the effectiveness of a media system is the level of satisfaction ex- 
pressed by faculty members. 

Facility users were asked this question: "From your 

standpoint as an instructor, are you satisfied with the present 
use of the multi-media system? " Only 33. 3 percent so indicated. 
This percentage closely parallels the reported satisfaction with 
the training received. 

This evident dissatisfaction may be the result of faculty 
members' frustration in developing software materials, an 
activity for which they had little training and one which 
frequently required a great deal of their time. In addition, all 
faculty members mentioned their dissatisfaction with lack of 
release time for developing media software. 

^ i. Level of use. It is important to know whether 
the use of a media system is increasing or decreasing, and if so, 
at what rate. 

Faculty users and media system directors were asked the 
following question: "In your viciW, is the use of the system 

increasing, stabilized, or decreiasing at this point in the life of 
the teaching system? " They then were provided with four choices 
Steeply I nc reasing, Slowly Increasing, Stabilized, or Decreasing 
liilization. 

Of the respondents who answered this question, all but five 
percent indicated that the system was increasing in use. The 
five percent indicated that the use of the system had stabilized. 
Forty (40) percent of the respondents indicated that the system was 
steeply increasing. Fifty-five (55) percent indicated that the use 
of the system was slowly increasing. A breakdown of system 
type is shown in the cha.rt below: 



CHART K- LEVEL OF SYSTEM USE 



System Steeply Slowly Stabili^^ed 

Typ e Increasing Inc r easing U se 



Multi- media 


40% 


60% 


- 


Dial- Access 


50% 


50% 


« . 


CAI 


100% 


- 


- 


Language Lab 


l6% 


68% 


16% 


Educ. TV 


tm 


100% 





j. Costs. Since it is an essential element in 
effectiveness evaluation, a series of questions were asked 
attempting to estimate the cost of hardware and software used 
in the system. Costs were estimated for two types of software: 

(1) software published and commercially available and (2) that 
software which was developed locally on the campus. 

These questions dealt with cost of hardware and components. 
Media directors were asked this question: "What is your estimate 

in dollars of the cost per hour of student use of this system 
hardware?" Briefly, respondents were asked to estimate cost 
of installation of the equipment, amortize this original cost over 
estimated life, add annual cost for operating and maintaining the 
equipment, and divide this sum by the estimated hours of student 
use per year. As a formula it looked like this: 

Hardware _ Amortized Original + Annual Operating + Annual 

Cost Costs Maintenance 

Costs 



Student hours of use per year 



Estimating the system component cost was quite difficult and 
most system directors found it almost impossible to provide 
answers to the questions. Only two directors gave realistic 
estimates of the hardware cost. One was for an operational 
Dial-Access system, the other for an experimental CAI system. 

The Dial- Access system hardware was estimated to cost 
approximately 30 cents per student hour of use and the CAI 
system hardware approximately $10 per student hour of use. 

k. Purch ased software. Software, whether 
developed locally or commercially, can be a very significant 
cost in the use of media system. 

Media directors and faculty members were asked this 
question: "Take into consideration the cost of published materials, 

texts, library resources which are allocated for this course 
(assigned and research) and other information resources which 
are purchased from outside sources. Attempt to assign a cost 
per student hour for this purchased material. Consider life 
expectancy of library acquisitions, text material and other 
factors of cost per student hour of use. Compute the cost per 
hour of educational activity derived from published materials, 
that isj the gross cost divided by student time spent with the 
materials. This equals a cost per student hour of use. " 
Represented in formula it appeared this way: 

Purchased 

Software = Cost of Materials Published 

Cost Student hours of use during life expectancy 

The faculty and media system directors found it quite difficult 
to work through this computation of software costs. However, 
a number of estimates were made which appear in the chart below. 
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CHART - COST OF PUBLISHED SOFT WARE /STUDENT HOUR OF USE 

ii 

Software Cost 

System Type Estimate ^ 

$.30, .33, .20, .05, .10 | 

$.32, .08 ; 

$ol5 j 

There is relatively little variation in the estimate of the cost of 
published materials, the range being from 33 cents/hour of use to 
5 cents/hour of use. It will be interesting to look into the reason for 
this relatively limited variability. Possibly it may be due to the lack 
of variety in the types of materials which are commercially available. 

One caution must be indicated here. There is no estimate concerning 

available CAI software„ It is suspected that this form of software will 

be leasedo Were it for sale it would probably be more expensive than 

any of the software materials for which cost estimates were made. i 

!• Home-made software . The next area of cost was 
that of locally developed software materials and an estimate of the 

cost per student hour of use was made. Faculty members and media } 

directors were asked to make an estimate of the cost of the materials 
and time required to produce software materials locally. 

They were asked this question: "Please perform a similar com- 

putation for locally pr epared mater ials . Here again, efforts should 
be devoted to life expectancy of on- campus prepared materials. Com- 
pute the cost of materials; add to this the staff time (faculty and 
system personnel hours multiplied by the hourly salary of each con- 
tributor). Divide this sum by the number of student hours of use for ; 

the software material. This will equal a cost per student hour of' use. " 



95 



Dial-Access Laboratory 
Multi-Media Classroom 
Language Laboratory 
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In terms of a formula the computation was as follows: 

Material Cost + Staff Time x Hourly Salaries 
Student hours of use during life expectancy 

Difficulty was also experienced in making an estimate of 
the cost of locally prepared software. The estimates made appear 
below. 

CHART M - COST OF LOCALLY PREPARED SOFTWARE/ 

STUDENT HOUR OF USE 

Local Software 
System Type Cost Estirr^ate 

Dial-Access Laboratory .50 .31 

Multi-Media Classroom 1.22 1.10 

Language Laboratory .40 

Here we find considerably more variation in the cost estimates 
than with published software. However, in addition, by comparing 
the locally developed software cost estimates with the published soft- 
ware estimates, we see that there is a tendency for the locally de- 
veloped materials to cost more than the commercially available 
materials. For example, it was reported that multi-media class- 
room materials, which were commercially available, cost approxi- 
mately 32 cents per student hour of use, whereas the locally developed 
materials cost $1. 22. The reasons behind this are worthy of investi- 
gation with the NEW FIELD, 

m. Gross, net and percentage of use. It was 
important to determine the percentage of utilization that systems are 
receiving. Several questions were asked of media directors: 

"Estimate actual net number of hours of student use per annum for 
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this system (compute average positions used X hours used per 
week X weeks used per year). " "Indicate the gross number of 
hours the system is available for use during the year (student 

positions available X hours scheduled per week X weeks scheduled 
per year). " 

From ansv/ers to these two questions, an estimate 'was made 
of the percentage of utilization of the system by dividing the total 
number of hours that the system was available for use into the 
total number of hours the system was actually used. Data gathered 
from these two questions appear in the chart below. 

CHART N - PERCENTAGE OF UTILIZATION 



System Type 


No. of Hours 
Available (Gross;) 
per annum 


No. of Hours 
Actual (net) 
per annum 


Percentage 

of 

Utilization 


Language Lab. 


186, 480 


113, 000 


66.0 


Language Lab. 


120,744 


91, 350 


75.7 


Language Lab. 


237, 864 


237, 864 


iOO.O 


Multi- Media 


172, 832 


137, 280 


79.4 


Dial- Acce s s 


105, 300 


28, 000 


26. 6 



Clearly, all the systems were being well used. Even the ap- 
parently low 26 percent reported at the Dial-Access Laboratory is a 
very respectable figure considering the fact that this laboratory was on 
a demand basis (open 7-11, walk in anytime) at a commuter -type 
university (peak loads when students are on campus, radical fall- off 
during slack hours). 
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II. 



FIELD STUDY 



C. Discussion 

In a previous section entitled Fi3idings, the experience 
in administering the FIELD was reviewed. The following 
discussion describes the changes which should be made in the 
present FIELD based on this experience. 

1. Definition. Several terms such as media, system, 
and functional specifications should be clearly defined at the 
time they are introduced in the NEW FIELD. Respondents should 
be asked to answer questions in terms of their experience with 
the media system under study unless otherwise specified. This 
will give the respondent a frame of reference which was lacking 
in the earlier edition of the FIELD. 

As indicated earlier, sketches of systems varied from 
those which showed interactions of equipment to those which 
showed interaction of students with equipment. After drawing 
the sketch of the media system, a revised FIELD item should 
request that the respondent indicate what type of interaction is 
shown on this sketch. 

The FIEUD in its tryout form asks questions about media 
systems in general wihtout regard to specific types of media 
systems. It is obvious from administering the FIELD that such 
an approach is not feasible. Each item on the FIELD has been 
identified as it applies to each system type. 

2. Te acher inclusio n. The FIELD in its revised version 
should consider the issue of the teacher as a part of a 

media system. Those areas where we are interested in the 
reaction to the media system (excluding the teacher) should 
specify that the teacher is not to be considered part of the system. 
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This inclusion of the teacher as part of the system configuration 
is something that has been overlooked in designing systems by 
many e due ato r s . 

3 . Developmental vs. operational. Questions should 
be added to the NEW FIELD to determine if respondents view the 
particular system under study as one in a developmental stage or 
an operational stage. In addition, respondents might be asked to 
specify whether the system design has changed during the develop- 
mental stages of system growth and to provide a description of how 
the system has evolved. Where appropriate results of the FIELD 
study on a larger sample of institutions should be interpreted in 
terms of the developmental and operational stages. 

4. System choice. Information concerning choice of 
media system courses in preference to conventionally taught courses 
must be gathered in part from hypothetical questions. An example 
might be: "Would you choose a media system taught course over a 

conventionally taught course, all other things being equal? " This 
approach is necessary, because students do not know, prior to enter- 
ing the course, whether it is taught by conventional means or through 
the use of a media system. A by-product of the study may be a 
recommendation to indicate in class catalogues which media are used 

in which courses. 

Attendance constraints. The question regarding at- 
tendance constraints needs improvement. It should be phrased so as 
to capture the subtle differences between the instructor's and student’ 
perception of compulsory or voluntary attendance. The revised ques- 
tion should determine, where, on the continum of attendance constraint 



each media system falls. 



Utilization data for each system should be interpreted differently 
for those courses which are voluntary and those courses which are 
compulsory. 

6. Costs. Questions about costs will have to be further 
refined to include very closely defined elements or cost factors which 
are to be included in the cost/effect ratios in the NEW FIELD. 

A detailed cost estimating procedure should be worked out per- 
mitting differential assessment of cost for each class of system. 

Subsets of questions and supporting explanatory paragraphs must be 
prepared to assist respondents in estimating and calculating all of 
the various costs which are involved in the purchase and use of a 
media system. Without such detailed methods for "forcing" the cost 
analysis, the NEW FIELD administration will fail to meet its evalua- 
tion goals. (It should be pointed out that the root word in "evaluation" 
is "value. ") 

7. Open-ended questions . Open-ended questions may be 

added to permit free discussion of likes and dislikes relative to the 
media system. Numerous respondents in the FIELD trial found it 
difficult to answer some questions either "yes" or "no." A question 
might be revised to read: "Are you generally happy with the per- 

formance of the hardware in this media system?" Space for comments 
after the yes/no answer should be provided. 

8. Duplication . Duplicate questions should be removed 
unless the duplication serves the purpose of checking the reliability 
of responses. 

9. FIELD structure . From the FIELD trial it is obvious 
that the questionnaire should be structured according to data require- 
ments (see page 108) rather than by respondent category. This revised 
structure is discussed more fully in the next section. 



II FIELD STUDY 



D. Specifications for a NE W FIELD 

The respondent oriented version of the FIELD instrument 
that was used in the tryout phase provided a vehicle for examination 
of a great many variable s„ It was fact-oriented and attempted to 
get at "ponderables. " It avoided rating scales and estimates of 
attitude. A revised FIELD instrument should permit examination 
of a smaller number of variables and determination of specific 
numerical values for each of the variables. 

In the following discussion a "NEW FIELD" is described 
which can be expected to yield numerical evaluative ratios. These 
then aie the specifications for a revised evaluation instrument. 

1. Structure of revised FIELD . The present set of 
questions will be rearranged and organized to yield answers to 
four basic questions. These basic questions, which can be answered 
with respect to any class of system, are: 

o What does it cost? 

o How much is it being used? 

Do users accept it as a means on instruction? 

° Do users learn through the use of it? 

Each of these basic questions can be expanded into an 
ordered sequence of questions which will yield a single class of 
facts: 

° Costs 

° Utilization level 

° Acceptance rate 

° Achievement units 



2. S ystem model . These four questions (cost, 
utUi:^;ation, acceptance, and achievement) can be answered in 
the context of the four basic System. Model components: Teacher, 
Software, Hardware, and Learner. These components can be 
arranged as follows: 






which cz .1 



This organization of questions will yield informadon 
be represented in a four-l3y~four chart. Such a chart 



is shown on the next page. 



Each of the 16 cells of the chart represents one class 
of information about a specific component of a system. For 
example, one cell of the chart indicates that cost delta can he 
collected in the new FIELD relative to expenses incurred through 
use of faculty in the system. Some cells of the chart have been 
left blank because that class of information about that system 
component is inappropriate, redundant or less germane to the 
purpose of tlic clcits* collection© 

3 ^ Classes of questions . The first class of questions 
(dealing with costs) will produce information about a larger cate- 
gory of data concerned with the antecedent conditions existing in 
each of the system components. The other three classes deal 
with information about criterion measures of system effectiveness 
The data collected about each system component provide informa- 
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tion regarding the relationship between antecedent conditions and 
criterion measures. This relationship forms the basis of the 
evaluative strategy. 

Relationships between cost as an antecedent condition 
and the criterion measures of system effectiveness (utilization, 
acceptance and achievement) will be reported in the form of 
ratios. Cost will be reported per unit of system effectiveness. 

The ratios to be computed are: 

a. Cost per_unit of use m ight be stated in dollars 
per student hour or cost per year of optimum use. The first of 
these would be calculated by summing cost of faculty, hardware 
and software, and dividing by the total student usage in hours. The 

ratio would appear as follows: 

Faculty Cost + Hardware Cost + Software Cos ^ 

Number of Students Average Hours of 

X Use per Student 



b. Cost ne r unit of acceptance might be stated in 
dollars per percentage point of faculty members using the system 
or students indicating they would enroll in another media-taught 
course. For example, the first can be shown as the ratio: 



Facu ity Cost + Hardware 
Percentage of ' 



Cos t + Software Cost 
)tance 



c. Cost per u nit of achievement might be stated in 
dollars per credit hour or grade point. The first of these can be 
shown as the ratio: 

" + Hardware Cost + Software Co^ 



These cost figures should be reported separately 
for each class of system. Because of the wide variations in cost 
from one system class to ainother, it is not meaningful to report 
a single cost figure for all media system classeso 

Relationships between system component ca- 
pabilities and system effectiveness will be reported in a variety 
of ways. 

The capabilities of the system components will be 
analyzed as a discrete -variableo For example^ system personnel 
will be asked whether the system permits feedback to the student 
participant. Answers can be used to identify those systems that 
do permit feedback and those which do not. 

Furthermore, faculty should be asked to indicate 
whether use of the system is increasing or decreasing. Answers 
can then be used to identify those classes of systems for which use 
is increasing and those for which use is decreasing. 

Dichotomous variables (like "presence/absences of feedback” 
and "increase/decrease in use") can be placed in a two-by-two 
contingency table and analyzed using a chi-square statistic. Re- 
sults of such an analysis might indicate that a significantly larger 
number of those systems which permit feedback are increasing m 
use than those which do not permit feedback. These types of data 
will permit statistical analysis only where data can be treated 
across systems. Any item of data such as quality of visual image, 
which is system specific will not permit statistical analysis 
because of the smallness of sample size. Statistical and narrative 
descriptions should be reported in these caseSo Descriptive statistics 
should include means, modes, ranges, frequencies and percentages. 



4, Classes of sy s tems . Eight general types or clus 



ters of systems should be studied. These are represented in the 
chart on the next page. 



P^od^cts . The end products of the NEW FIELD 

would be; 



Information based on the combined experience 
of a sample of 30 colleges and universities in the use of a large number 
of .systems. This inforndation will be collected in four categories: 



o Cost 
o Utilization 
o Acceptance 
o Achievement 

as they relate to the system components; 
o Teacher 
o Software 
o Hardware 
o Learner 

for each of eight classes of systems; 
o CAI. 
o LL 
o DAIRS 
o MM 

o Audio- Tutorial, 
o CCTV 

o Open- Circuit TV 

o VTR {used in micro -teaching 
applications). 



CHART p - SYSTEM CLASSIFICATION 



System Classes 



Type of System within Classes 



VERBAL, LINEAR 
PRINTED 



VERBAL, LINEAR, 
AUDIO 







MULTI -SENSORY, 
LOCAL 



MULTI-SENSORY, 

DISTRIBUTED 



COMPLEX ACTIVITY 
DEVICES 



C A I (Computer Assisted Instruction 






L L (Language Laboratory) 





DAIRS (Dial-Access-Information 
Retrieval System) 



Fixed MM Classrooms 



aagi:Bsggaga 






Portable Audio -Tutorial Applications 






Closed- Circuit TV Systems 



a ' p. ■ -Jg Jg— 



Open-Circuit TV Systems 
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6o Specifications (Forms ). The NEW FIELD should 
consist of approximately 10 separate data collection forms. Each 
form should collect one class of information about media systems. 
The following is a list of the specific data collection forms; 

a. System Definer Form 
h. Media Program Profile 

c. Hardware and Facility Cost Estimation Procedure 
do System Utilization Data Form 

e. System Use and Growth Forecast 

f. Software Cost Estimation Procedure (Appendix B) 

go Personnel Cost Estimation Procedure 

h. System Benefits Measurement and Reporting 
Form 

io Facility Skills Inventory 

j. Large Sample Group Administration Instruments 

k. Case Study Report Form (System Profile) 

The following breakdown describes the purpose of each 
data collection form which should be developed and gives samples 
of items appearing on each form together with a brief discussion 
of the data, collection procedures to be used v/ith each. 

ao System D efiner Form 
(1) Purpose 

Data collected on this form will provide a 
detailed description of each of the systems under study. From 
this description the system will be classified. Classification 
will permit like systems to be grouped by system type. Data can 
be aggregated and reported by system classification. 
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(2) Sample questions 

(a) What forms of input to students are used 
in the most typical application of this system? What forms of in- 
put are possible with this system? (Forms of input include micro- 
fiche, work sheets, tape, film, slides, video tape, etc. ) 

(b) What response options are used with the 
most typical application of this system? What response options 
are possible but not typically used? (Response options include 
pushing buttons, turning pages, recorded utterances, hand written 
notes, etc. ) 

(3) Procedures 

Other specific hardware characteristics 
(such as those found on pages 15 through 18 of the Technician 
Questionnaire of the Old FIELD instrument) will be collected on 
this form. Much of the data can be obtained by listing documents; 
other facts related to the definition of the system can be obtained 
from the system director and technicians. 

b. Media Program Profile 

(1) Purpose 

As the previous section attempted to define 
the media system under study, this form strives to identify the 
characteristics of a larger factor, the media program of the 
institutiono 

(2) Sample questions 

(a) Was this media system installed to 
solve an existing problem or satisfy an existing need? 

(b) Have functional and technical specifica- 
tions been written for the hardware and software components of 



of this system? 

Other questions can be adapted from the "Evaluative 
Checklist, " an instrument for self-evaluation of media programs 
by W. R. Fulton of the University of Oklahoma at Norman. 

( ^ ) Procedures 

This information can be obtained from 
the system director and in part from the system technicians and 
student users. Much of it can be obtained in interviews and 

questionnaire formats. 

c . Hardware and Facility Cost Estimation 

Procedure 

(1) Purpose 

This is one aspect of total system cost 
on which data can be gathered. 

(2) Sample questions 

(a) What is the replacement cost for each 

item of hardware in the present system? 

(b) What is the estimated cost of the 
physical facility occupied by the system hardware (estimation 
procedure provided) ? 

(3) Procedures 

This information will be obtained by 
interview and by searching of documents maintained by the media 
system staff and Building and Grounds Department of the university. 

d. System Utilization Data Form 
(1) Purpose 

An important measure of the performance 
of cl media system is the degree of use it receives. 



(2) Sample questions 



(a) How many hours per week do students 
use the media system beyond the required use? 

(b) How many times on the average does 
a student review a program of instruction? 

(c) Is the use of the media system in- 
creasing or decreasing at the present time, among students and 
among faculty members? 

(d) What is the student load which the 
system is carrying? (that is, number of students using the system 
on a weekly basis multiplied by the number of hours of use for each 
student - paradigm furnished.) 

(3) Procedures 

Information will be obtained by individual 
administration of a questionnaire to system directors and faculty 
members and by group administration of a questionnaire to classes 
of students using the media system. 

e. S ystem Use and Growth Forecast 

(1) Purpose 

Use of this data form will gather informa- 
tion concerning the forecast utilization level and growth of system 
capacity. 

(2) Samp le questions 

(a) Looking forward one year, will the 
intensity of use increase or decrease? Please give percentage 
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factor of increase. 



(b) Looking forward one year, will the 
size of the system be smaller, larger, stay the same? If 
larger, specify percentage growth. 

(c) Looking forward five years, will the 
intensity of use increase? By what factor? 

(d) Looking forward five years, will the 
size of the system be larger? By what factor? 

(3) Procedures 

Forecasts will be made by administrators, 
faculty members, (presently using media systems) and by the 
system director. 

f . Software Cost Estimati-on.Procedure 

(1) Purpose 

This instrument gathers another component 
of the total system cost. 

(2) S ample que s tion s 

(a) What is the cost of software purchased 

outside annually? 

(b) What is the cost of inhouse produced 
software (cost accumulating procedure provided) ? 

(3) Procedures 

Faculty members and system staff are 
the best sources for this information. The figures arrived at 
can be plugged into an included mathematical formula to produce 
the software cost. 
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Personnel Cost Estimation Procedure 



(1 ) Purpose 

A major component of th.e total system 
cost is gathered by this data form. 

(2) Sample questions 

(a) What is the total cost of your technical 
staff (procedure for accumulation and estimating various costs)? 

(b) What part of the faculty user cost can 
be attributed to the operation and maintenance of the system? 

(c) What part of faculty member time is 
used to produce system software? 

(3) Procedure s 

System director, and in some cases 
faculty users, will be the best sources of information for this 
form. Some sources of documented data may be available. 

h. System Benefits Measurement and 

Reporting Form 

( 1 ) Purpose 

Measurement of the acceptance the media 
system has received among faculty members and students is the 
purpose as well as measurement of achievement among students 
which can be attributed to the media sy stem. 

(2) Sample questions 

(a) What percentage of students using 
media systems attend regularly? 



(b) All other things being equal, would 
you choose a media system taught course over a conventionally 
taught course? 

(c) Do you find that the functional require- 
ments of the teaching task are met by the media system? 

(d) How many credit hours of instruction 
can be attributed to media instruction? 

(e) Do a greater percentage of students 
in media taught courses complete instruction than in conven- 
tionally taught courses? 

(3) Procedures 

Data for this section will be gathered by 
the use of questionnaires administered to students in groups and 
faculty members, administrative officials, and system staff 
individually, 

io Facul ty Skills Inventory 

(1) Purpose 

This data form gathers information 
relative to the skills media users feel faculty members should 
possess to make efficient use of media systems and the degree 
to which they believe that faculty members at the institution under 
study ha.ve received training to develop these skills. 

(2) Sample questio ns 

(a) What skills do you feel are required 
to make efficient; use of media systems? 



(b) Have you received training in the 



use of- media systems? 

(3) Procedures 

Faculty members and media personnel 
will be administered this questionnaire individually. The work of 
Ann Martin at the University of Pittsburgh will be drawn upon in 
preparing the instruments. 



h Large Sample Group Administrat ion 
Instruments 



(1 ) Purpose 

This instrument gathers data concerning 
the attitudes, interests, expectations, likes and dislikes of student 
in regard to media systems. 



(2) Sample questions 

(a) What do you like about mediated 



instruction? 



(b) What changes are required to 



improve instruction? 



(3) Procedures 

These questionnaires will be administered 
to groups of student users in classroom situations. 



k. Case Study Report Form (System Profile) 

1. his short form will be used to accumulate 
the gross statistics from the other NEW FIELD documents and to 
calculate the derivative statistics listed in the section entitled, 
"Products of NEW FIELD . " 



7. Normative data 



While the complete list of specific facts to be 
generated by the NEW FIELD cannot be specified in its entirety, 
the following facts are typical: 

a. System cost - estimated costs to replace 

existing system at today's (Fall 1968) prices 

Space costs - gross annuaJ costs of facility 
in which the system is operated 

Use level - estimated annual utilization of 
the facility in student hours 

d. Library size - size of existing software 

library in hours 

Personnel costs - gross annual personnel 
costs for system operation exclusive of 
student study time 

fo Achievement ir^asur es - 

® Gross - credit hours earned in courses 
using the system 

credit hours attributable to the use 
of the system 

go Acceptance levels 

A number of derivative statistics will then be 
calculated for individual systems and classes of systems. Exampl 
of these statistics are: 



© Cost per student hour of study 
© Cost per credit hour earned 
© Original equipment cost per hour of use 
® Maiintenance and operating cost per hour of use 

@ Software development cost per hour of use, etc. 

h 0 Non- numeric data 

Two forms of non-numeric data can be reported 
as a product of the NEW FIELDo One will be a narrative descrip- 
tion of characteristics of the system in each of the classeso The 
other will be statements regarding important issues in the media 
field. These issues may include: 

(1) Faculty release time for software 
development 

(2) Overcoming resistance through familiarity 
and training 

(3) Copyright ownership as an incentive 

(4) Does the systems approach "work"? 

(5) Need for administrative support, etc. 

8 . Evaluation means 

The published results, together with published 
instrumvents used in the dat a gathering phase, should provide 
other colleges a basis for self-evaluation. 

9c Planning uses 

Other institutions considering new media systems 
(and interested in choosing among alternative learning systems) 
may use the suggested instruments in their planning activities 
in such a way as to predict cost and determine effects in advance^ 
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:!d instrument for the Evaluation of Learning Devices 

^DUCTED BY THE AMERICAN INSTITUTES FOR RESEARCH 



RESPONDENT 

IDENTIFiCATiON: 



NAME 




(USER OF MEDIA SYSTEM) 



LEARN iNG DEVICE 
IDENTIFICATION: 




This questionnaire is part of a larger study- 
conducted by the American Institutes for Research in 
which Faculty, Staff, and Students are being asked to help 
e'valuate some of the Multi-Media Teaching Devices in use 
on this campus. Your responses will be merged with the 
responses of others; -your ideas and reactions will be 
treated confidentiall-y; and no personal identifications 
will be included in reports concerning the stud-y. 



o 




1 




i-'yvANr-iJj 1 



hJ-i *-= 



CAI = 



DAIRS 



MM == 



ETV 



VTR 



o 
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LANGUAGE EAB 




Many booths or desks 
with earphones for 
listening / practicing 
of languages 



COMPUTER A SSLS TED 
INSTRUCTION 




Tele-Typewriter console 
for input and printout 

communications with 

shared-time computer. 



= DIAL ACCESS INFORMATION 
RETRIEVAL SYSTEM 




Dial or push button 
for each student to 
use in selecting and 
using audio-tapes 
( and som.etimes visuals 
or video signals). 




Classroom with 
screens, speakers, 
projection equipment 
for complex audio- 
visual. presentations 
Sometimes .with response 
buttons at student seats). 



Television is used to 
distribute audible &; 
visible information to 
one or more remote 
locations . 



= VIDEOTAPE 




A TV camera and tape recorder 
are used to capture students' 
audio and visual performance. 
Playback provides students , 
information needed to improve’ 
later performance. 






j 



GENERAL BACKGROUND INFORMATION 



Name : 



Sex : 



Age; 



Marital status; 



Number of dependents 
(Exclude yourself) 

Student status; 



Atte-ndence Level; 



Residence Status; 



Major; 




□ Freshmg.n 

I I Sophomore 

□ Junior 




1 1 Senior 
1 1 Graduate 
I □ Other (desi 


gnate) ' I 


~1~1 Full-time or 
1 (Z) Part-time 


(yourself) 


r~] Commuter 


Which do you consider I 


l~~] Resident 


yourself to be? J 




I Major(s) 


Minor . . 




low many hours per week do you spend on 
educational activities? 



lo you work? 

If yes, how many hours per week? 



In laboratories 

(Media and other) 



Classtime 



All other 




(Study, library, research, 
extra-curricular, etc. ) 



Total Hours 




o 



Which, if any Media Instructionax 
System do you use in your 
academic work? 



□ LL 

□ CAI 

□ DAIRS 



Cl. MM (See preceeding 
r — 1 panel ol materiail 

I — I ETV describing these 

□ VTR systems) 



The following aeries of questions explores 
student use of time and student attitudes toward time 
use. 



STUDENT USE /TIME 



During an average week how many hours 
do you spend using media beyond the 
scheduled class hours? (If offered only. 

during class, how many hours durhag class 
do you use media per week?) 

In general, do you have an opportunity to 
review or repeat your use of instructional 
programs ? 

If yes, how many times can you review 
or re-use the same material 




‘ VlM*inri *~i[ illil«i Ml ^ IIWI i< »< «■— ■■mi I>III iMiai I — I^ln■^ 

number of times 



How many times d£ you review the material? 



Do you think saving your time is an 
important element to consider when 
choosing methods of instruction? 



Do you think saving your time is an im- 
portant element to consider when choosing 
methods of instruction. 





number of times 


^ — - - - ; — r — ^ — 1^.— — . I ■ 111 


□ Yes 


□ no 




□ Yes 


□ No 



-oThese questions explore the availability and type 
of orientation to use of instructional systems. 



What kind of training were you given in the 
use of media ? 



.„ERIC 



TRAINING 



ClNone 

m 

ClWritten instruction 
1 IPemonstrations 
□s upervised practice 
n Verbal briefing 

(Check all applicable) 



•') a 







■f you experience difficulty during 
:he time you use this instructional 
system, what type of assistance; io 
■available.? 



f — 



[~] Instructor . ' 

□ Proctor 

I I Equipment advisor 

□ O ther (specify) 









The following questions explores the affective 
elements involved in the use of instructional 
systems . 



Does the institution periodically assess 
students' attitudes? 



toward the subject: 



toward the medium 
of instruction: 



□ Yes Dno 

d] Unknown 

□ Yes Dno 

EH Unknown 



J 



Does the institution periodically assess 
the students' interest? 


in the subject: 

in the medium of 
instruction: 


, U Yes □ No 
, Unknown 

□ Yes GNo 

n Unknown 










Lf yes, does the institution make use of 
these data to modify programs 


□ Yes GNo" 


( iUnknown 





The number of system-taught courses you 
;e and the fact that you choose them may tell 
something about the level of acceptance of 
lated instruction at your institution. 



NUMBERS CHOICE 



the same course were being offered two 
ays (one using conventional methods & 
ie using mediated instruction) which 
ould you choose, all other things being 
)[ual? 



Have you used more than one multi-media 
sy.sleni f>f instruction? 




L 



O 

ERIC 



0 



Jil 






If yes, which instructional system do 
jfou prefer ? 



1— J 

□ CAI 

□ DIARS 

[I] Other 



□ MM 

□ ETV 

□ VTR 



(See panel for explanation) 






Would you choose another course taught by 
the mediated instructional system approach? 

Wh-at reasons can you give for choosing 
a course taught by a system of mediated 

instruction? 



Did you have a “choice" (to participate or not 
to participate) in this particular media 
program? 

Which type of courses are taught 
with this system? 



□ Yes □ No 



□ Time of day 

r~~l Required course -- no conventionally 
taught sections open 
I 1 Professor 

□ Type of system 

I 1 Other 




□Yes DNo I I Unknown 



Required[i| O Elective nUnknown 



the quesliions belov; examine the effect 
of faulty systems on the learning 
process . 

Do you find that equipment breakdowns 
are a problem? 

Is equipment generally available when you 
want to use it? 



Do inadequacies in the equipment seriously 
disturb learning? 



RELIABILITY & SCHEDULING 



□.Yes 


□ No, 




□ Yes 


□ No 




□ Yes 


□ No 
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What type of problems have you experienced 
when using media systems ? 



The next set cf questions reveals ways of 
measuring the effectiveness of instructional 
systems 



n In sufficient time | 

d Time consuming. 

I I Disciplinary (noisy, in- 
attentive) 

□ Media content not related I 
to course content 

(HI Availability 



( I Equipment breakdown 
. [~1 Other 



EFECT 



re you tested on your knowledge or skill 
po.n entering a system instructional program? . 



7hen leaving a system taught course are you 
neasured on your acquired knowledge or skill? 




lease check those measures 
f knowledge or skills which 
re used: 



- — ' 


At the beginning 
of the course 


1 At the end of 
the course 


Multiple choice or other 
objective tests 






Essay -type or other 
subjectively evaluated 






Oral or less structured | 
methods | 


1 

i 


• 

1 


Other (designate) I 
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lat kind of tests are used? 



you find the measures used to be the most 
jctive way to evaluate achievement? 

>s the institution use performance tests 
an indicator in miodifying the instructional 
tern? 



you find the use of media systems as 
tructional tools increasing? 

If yes, which factors would you say 
re contributed to the increased use of the 
dia systems ? 



hese questions attempt to get at problems 
iciated v/ith lost time in conventional instruction, 
'er these questions on the basis of a course you 
taklrig this semester which is not taught with 
use of instructional media devices. 



Q Unknown 
D Standardized 
f~l Locally devised 






" I 

I 



i ; 
• 1 



Yes 



No ■ 



Unknown 







I I Increased familiarity with system 
r~1 Growing availability of system 
r~] Instructors increasing interest 
C3 Additional time for staff to develop 
materials 

D Other 




CONVENTIONAL INSTRUCTION 
DOWN-TIME 



uring your last class meeting in a conventional 
ituation, could the instructor have covered 
le material in less time? 

If yes, what is the: 






□ Yes n No 



full length of the session: 



actual time required: 



possible savings in time: 
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Concerning this same conventional instruction 
course, hov^ many lectures are cancelled 
by the instructor ? 



How frequently is the instructor late for class? 



If 'he instructor is sometimes late, 
what is the average length of his late 
arrival for this course? 

Is there a rule (written or unwritten) on this 
campus regarding hov/ long a class should wait 
for a tardy teacher? 

Comment. 



It 1:5 sometimes useful to examine the value students 
place on a college education in terms of financial 
expense and time committments they make to it. These 
questions pursue this line of reasoning. 

Please estimate the gross cost per year of 
your college education (consider tuition, 
room and board, books , clothing, trans- 
portation and all other expenses incurred). 



jG:2S:e5,' 

[ Cancellations 
□ Ve 



ry few 

□ 1 in 10 

□ 2 in lOQjsjone 

□ 3 in 10 
I I more 

[Zl Very infrequently 
D 1 in 10 

Q 2 in 10 r—j 

□ 3 in 10 □ Never 

□ more frequently 





EDUCATIONAL EXPENSE 




$ 

Gross cost per year 




How is this expense borne? Use fractions 
to indicate the portion of the total expense 
paid: 

(1) by you ( from earnings and savings 

(2) by your family (cash, loans, etc.) 

(3) by the college or society in general 

(subsidies, tuition reductions, 

scholarships, and other support). 
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My contribution 




Family contribution 




Other support 




y 



u 




I'low many days are there in the academic 
year at your institution (deduct vacation, 
holidays, weekends, etc.) and figure out 
the approximate number of days per year 
you devote to educational activities) ? 



How many hours a day (on the average) do you 
devote to direct "curricular" educational 
activities (in class, studying, research, 

library, etc.)? 

Hov 7 many hours a day (on the average) do 
you devote to extra-curricular but significant 
collegiate activities (college-related but 
not recreational) ? 




I days 

academic year 




curricular hours 




additional hours 
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d Instrument for the Evaluation of Learning Devices 

IDUCTED BY THE AMERICAN INSTITUTES FOR RESEARCH 



RESPONDENT 

DENTIFICATION: 



INSTITUTION 




(At Large) 



LEARNING DEVICE 
IDENTIFICATION: 



— 

I nstitution 

□ CAI □ ET V I 

? 

□ D AIRS □ RL I 

i 

□ MM CLASSRMD microteaching I 

j 

! 

i 

■ .. — — — - - '■■ ■■ i 

j 

i 

! 

J 




This questionnaire is part of a larger study 
conducted by the American Instifaites for Research in 
which Faculty, Staff, and Students are being asked to help 
evaluate some of the Multi-Media Teaching Devices in use 
on this campus. Your responses will be merged with, the 
responses of others; your ideas and reactions will be 
treated confidentially; and no. personal identifications 
will be .included in reports concerning the study. 



o 
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STUDENT 



INDEX 



GENERAL BACKGROUND INFORMATION 

MEDIATED INSTRUCTION 

AFFECT o . . . . 

EFFECT • 

INSTRUCTION DOWN /TIME 

A 

EDUCATIONAL EXPENSES 



Name: 



GENERAL BACKGROUND INFORMATION 



Sex: 

Age: 

Marital status: 
Number of dependents 
Student status: 



Major: 



How many houi's per week do you spend on 
educational activities? 




CD Male 



j 



CD F emade 






CD Single 



□ 



Married 




Ao CDPreshman 
CD Sophomore 
□j uni or 
CD Senior 
CD Graduate 
CDother (designate)^ 

B. CDPull-time or 
CD Part- time 

C. Q Commuter or 

CD Resident 



Major(s)_ 

Minor 



arcsarS 




jt-'yviMJ'.iJj 1 



LL LANGUAGE LAB 




Many booths or desks 
with earphones for 
listening / practicing 
of languages 



CAI = COMPUTER ASSISTED 
INSTRUCTION 




Tele-Typewriter console 
for input and printout 

communications with 

shared-time computer. 



DAIRS = DIAL ACCESS INFORMATION 

retrieval system 




Dial or push button 
for each student to 
use in selecting and 
using audio-tapes 
( and sometimes visuals 
or video signals) . 



MM = 



ETV = 




Classroom with 
screens, speakers, 
projection equipment 
for complex audio- 
visual presentations 
(sometimes with response 
buttons at student seats) . 



Television is used to 
distribute audible Sc 
visible information to, 
one or more remote 
locations . 



VTR 




A TV camera and tape recorder 
are used to capture students’ 
audio and visual performance. 
Playback provides students 
information needed to improve- 
later performance. 
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Do you work? 



If yes, how many hours 
per week? 



Do you use any Media Instructional 
System in your academic work? 



Have you heard any of these comments 
about mediated instruction courses at 
this institution? 




MEDIATED INSTRUCTION 



□ 


LL 


n 1 

□ MM 

1 1 


□ 


CAI 


□ ETV i 


□ 


DAIRS 


□ VTR 




(See preceeding panel of material 




describing these 


teaching systems) 



I 



□ "The t e aching is planned better" 



□ "You don’t have to attend class 
regularly" 



,n 



□ "The professor is more effective 
O "The professor 'doesn't come across 



I ft 



□ Too many people for the available 
equipment. 



□ No communication with Professor 

0 Poor sound and/or image quality 

1 I Content not related to tests 



2 ^ 

^ if 




fi 




r 



In your opinion is the cost of an 
instructional system too great in relation 
to the effectiveness of such systems? 

Or is the cost "worth it" in terms of 
speed or ease of learning? 



I I Yes, costs 
too much 



ED No, worth 
what it costs 



□ Unknown 



In your opinion, do students using 
instructional systems consume 
more time in study than students 
in conventionally taught classes ? 
Or do classes taught with media 
systems use less student time? 



The following questions explore student feelings 
concerning the use of various instructional methods. 




Does your institution periodically 
assess the students' attitudes? 



Does the institution periodically 
assess the students' interest? 



Does the institution make use of this 
data to modify programs? 



toward the subject: ED Yes ED 

□ Unknown 

toward the instruc- 



tion: 


□ Yes DNo. 
Unknown 


. « . « . 


in the subject: 


Yes QNo 




Unknown 


' in the medium of 


YesD NoD 


instruction: 


Unknown 




U Yes □’ No 


ED Unknown 

■ » ■ 



Do you feel that your attitude towards 
a particular training medium is an 
important element in learning 
effectiveness ? 
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EFFECT 



The next f.ot of questionr. rovca]i:< waye of 
measuring tlie effect ' vcnei>a of .Inciti’iict. Tonal 
systems. 

{Exclude standardized tests used as college 
entrance or placement purposes. ) 



Are you measured on your acquired knowledge 
or skil before completing courses at this 
institution ? 



□ 


Yes 


□ No 














□ 

■■ - V - 


Yes 


□ No 





; 

! 



What measures of knowledge 
or skills are used? 



- 


prior to 
entering 


after the 
course 


Multiple choice o/r other 
objective tests 






Essay-type or other 
subjective -type 






Oral and less structured 
methods 






Other (designate) 








Do you find the measures used 
to be an effective way to evaluate 
achievement? 




Do you think the institution uses 
performance tests as an 
indicator in modifying the 
instructional system? 




Do you find the use of modern devices 
and instructional media systems as 
teaching tools increasing? 
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Thof.o que:5tion'o attempt to at prob^oiiu; 

aaaociutod with i>>ot time in conventional Inr.truct j on. 
/Vnawci* these quest Iona on the basia of a course you 
arc taking thla aemester which is not taught v/lth 
the use of instructional media devices. 

During your last class meeting in a conventional 
situation, could the instructor have covered 
the material in less time? 

If yes, what is the: 



Concerning this same course taught by 
conventional media, how many lectures are 
cancelled by the instructor? 



How frequently is the instructor late for class? 



What is the average length of late arrival for the 
instructor in this course? 



INSTRUCTION DOWN /TIME i 

hi 



□ Yei 



□ No 



full length of the session: 



possible savings in time: 



sKTSkaa I 



actual time required 









Cancellations 

Dvery few 

□ l in 10 

□ 2 in 10 
Ds in 10 
CUmolre 




! 

I 

i 

I 




I 



I 



Is there a rule (written or unwritten) on this 
campus regarding how long a class should 
wait for a tardy teacher? 

Comment: 
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It lu ijomotlmos useful to examine the value students 
place on a college educate on in tex’ms of financial 
expense and time committments they make to it. These 
questions explore this line of reasoning. 



Please estimate the gross cost per year of 
your college education (consider tuition, 
room and board, books, clothing, trans- 
portation and all other expenses incurred 
"and include summer school if you attend. ) 



EDUCATIONAL EXPENSES 




How is this expense borne? Use fractions 
to indicate the portion of the total expense 
paid: 



(1) by you (from earnings and savings) 

(2) by your family (cash, loans, etc. ) 

(3) by the college or society in 
general (subsidies, tuition 
reductions, scholarships, 
and other support). 



S/Ly contr ibution 



Family contribution 




Other support 



How many days are there in the academic 
year at your institution (deduct vacation, 
holidays, weekends, etc. ) and figure out 
the approximate number of days per year 
devoted to educational activities ? (Include 
summer if included in cost estimate above. ) 



How many hours a day (on the average) do you 
devote to direct "curricular'* educational 
activities ? (Include study, classes, i. e. all 
educational activities. ) 



days 



academic year 




curricular hours 




How many hours a day (on the average) do 
you devote to extra-curricular but significant 

collegiate activities? ' additional hours 













fristruineni for the fv.iuotion of ■ sa.'ni:rc' Devices 

iUCTiSD BY eira .eefiTSTcvEf; roe Rv;SjiAriCH 



RtSPGNDEoT 

iDENT’FlCAf-Oi'O 



1 



1 Name 



Yr:!o 












‘r 









(User of (Vi M System) 



LEARNING DEVICE 
IDENTiFiCATiON: 



1 nstitution 



This questionnaire is pa.rt of a larger study ■ 
conducted by the ..A. me ri can Institute>s for T-esearch in 
which Faculty, StFiff, and Students are being asked to help 
evaluate so,rne of the Multi -Media Teaching Devices in use 
on this cs-mpus. Your responses will be merged with the 
'responses of others; your ideas and reactions will be 
treated .confidentially; and no personal identifications 
Vvill be included in. reports concerning the study. 




; f-' * Y m 



1 

V 



□ CAI □:ETV 

□ DAIRS □ DL 

.□ MM CTuASSRjvin MICROTEACHING 



J 






• : AV’U - 



■u- V b ■: V c ' Q.r. ultv A'f u-'.g; t.Ao t tuN ■UvA.tfC 



mM 

- V ■ ' 



TEACHER USER INDEX 



Faculty Description . . . . 1 

Student Load 2 

Prerequisites . . o ... . . o p . . . . o . 3 



Evaluation ^ 

Attendance Constraints 7 

Affect Measures . 9 

TeacKer Attitudes 10 

Effect on Instruction 11 

Leadership 14 

Fidelity Requirements 14 

Software Selection 15 

Software Costs «... 1^ 

Critique IS 



o 
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The folloviing series of questions gives a 
"thumbnail sketch" of the faculty. This refers to 
number and breakdov'jn of faculty time. 



What type of research is emphasized at 
this institution? 



r 



Please estimate the percent of time you 
spend on the following: 



□ behavioral research 

□ mathematical, chemical, 
biological, physical, etc. 

□ medical 

] I none 

% 

0 other (designate) 






under graduate instruction 

classroom preparation 

professional activities outside 
university 

— research 

academic administration 

counselling 

graduate instruction 

other 

100 % 



1 





Data on system use is reqviired in order to establish 
the usefulness of a system. This series of questions is 
to he answered hy selected teachers who use the system. 
Their answers should he cast in terms of those courses 
they are currently teaching, using the device under study. 

How many students do you have partici- 
pating in courses using this teaching media 
system ? 



How many hours per week (on the avervage) 
do they use the' system? 



The gross number of student hours/week is 

Please assume that this same course were 
being taught by conventional means. Esti- 
mate the gross time required to cover the 
same material with equivalent success in 
terms of achievement level. 



Calculate or estimate the percentage of the 
students in this teaching situation who attend 
regularly. 



STUDENT LOAD 



Mediated Instruction 



(number of students) 



(hours) for one student 



(calculate from above) 



I Conventional Instruction 

(number of students) 

(hours) for one student 
Gross student hours 



1 







B'/- ’ ’li V .■HtVr- V 






% 






. 


- . , -• 




■ ■ 1. • 



o 
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ftimes the prerequisites or "input hurdles" have a 
'ound effect on the output of a multi-media system. 

5 series of questions attempts to obtain information on how 
ients are selected (or select themselves) for courses taiight 
this multi-media system. 

hich type of courses are taught with 
is system? 

the course you teach using this system, 
iio taught without recourse to a multi - 
=dia teaching system? 



PREREQUISllES 



□ Yei 










□ NO 



you think students choose a course taught 
media systems over courses taught by I 
nventional inethods of instruction? 



□ Y 

'lease comment: 



es 



□ No n Maybe 





both conventional and mediated instruction 
e available, what percent of the total 
.mber of students would select mediated 
struction? 

i this course open to any student at the 
niversity ? 

If no, how are students selected for 
nrollment in this course? 
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□ By instructor interview 

Q By passing grade on prerequi- 
site course 

Q By simple developmental 

growth (for example, achieving 
sophomore status, regardless 
of the freshman course mix) 

□ College counselor approval 

□ Demonstrable academic need 
(failure of comprehensive or 
other test) 

Q Progression in curriculum 
Other entrance hurdle 




Vhich of the following student characteristics 
^re. considered when selecting a population to 
participate in a system mediated instructional 
rogram? 



tre intellectual aptitude scores (I.Q., etc.) 

L important variable in choosing students 
participate in system instructional programs ? 

If yes, what measures are used? 

If no, please comment. 



•e students pre-tested during the beginning 
a course to determine the level of knowledge 
skill prior to beginning the system instruc- 
»nal program? 

If yes, please specify the form of pre-test 
used. ^ 



)re~tests are used, do you believe they 
asure knowledge, skills, or aptitude? 



□ Age 

□ Sex 

CD Aptitude 

d] Interests 

□ Skills 

□ Knowledge 




□ Yes □ No 




[_] A final exam of prerequisite course 



serves as pretest for this course 

□ 

A pre-test is used to determine 
admission to this course 
Cl A pre-test is given to all registrants 
at the beginning of this course 
□ Oral or other informal evaluation is 
made at the beginning of the course 
□ Other (please specify)^^ 






□ knowledge 

□ skills 

□ aptitudes 



\ ERIC 
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tests are used to dis c riminat e a niong students 
re tViey marked "on the curve" or is theic a 
lastery standard which, if all could beat, all 
'ould pas s ? 



CIl marked "on the curve 
I I mastery standard 



M 



: tests are used to measure achievement, 
rhat kind of tests are they? 



f~l locally devised 
! I standardized 




[f you use periodic series of tests, how 
[requent are they? 

Do your tests measure Cognitive Functions? 
[f so which: 



Does your final evaluation bear any re- 
lation to your prerequisite or selection 
method? 

Do you feel tests written for the purpose 
of discriminating among students or to 
establish a standard of mastery can also 
be used to measure system effectiveness? 

If no, what form, style, manner of 
testing can be used to measure system 
effectiveness (as opposed to student achieve' 
ment) ? Describe. 




we 



ekly or less O quarterly D mid-te rn^ 



in Yes 


□ No ... . 


1 ■ r -1 

1 ' 1 Comprehension 


n Analysis 




1 1 Knowledge 


1 1 Synthesis 




1 1 Application 


r~1 Evaluation 




1 (See Bloom’s Taxonomy) 








□ Yes 


□ No 




' If yes, how related? 










1 □ Yes 


□ No 





I 




What tests are peculiarly adapted to 
administration through the instructional 
media system itself? 
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:his coarse taught by mediated instruction can 

compared with a course taught in a conventional Q Lower in system taught 
traditional. manner, is student attendance higher I Q] Higher in system taught 
lower than attendance in the comparable Q Not comparable 

aventional course? ■ 



t 




RegiJstration and attendance are not the only measures 

' use. It is also important to determine the proportion COURSE COMPLETION 

’ students who complete system taught courses. 



I 

f- 



the total number of students signed up to 
ke a course ta.ught by mediated instruction, 
xat percentage (estimated) remain in the 
urse? 



: those who do not remain in the course, 
lat percentage give "the system" as a reason 
r dropping out of the course? 






Estimated % completing course 



Fractional part of the above 
percentage not completing the 



course 








When a significant number of students selects 
additional courses (in the field to which they 
were introduced by mediated instruction) does 
this connote effective instruction in the first 
course ? 



Is student i nterest in a subject or career 
an important goal in your teaching ? 
Explain. 



How do you identify it? 



Are students’ interests measured? 

If yes: At input point? 

During process? 

After completion? 

If yeSj what interest measures do 
you use? 



If no, do you feel that interest can 
be measured before a student enters a 
system instructional program? 








I I Yes D No 



□ Yes 

□ Yes 

□ Yes 




The next questions examine affect, such as, interest, 
titude and attitude change in relation to the goals of the 
:aching system. 



O 
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IMPORTANT; This statement describes an important 
aspect of the matter under consideration. 

SIGN IF! CANT; This statement describes a significant, 
but somewhat less important aspect or element of the 
matter under consideration. 

NOT INFLUENTIAL; This statement may have 
bearing on the matter but is among the less 
influential issues. 



I 



! J. 
1 1 

' { 

’ ! 
•i 

TEACHER ATTITUDES j 

j, 



lease rate (see rating system on preceeding 
ge) the factors with respect to their impor- 
Qce to the increased use of this media system 
an instr^Tctional tool: 



I 

! 

1 

i 



i 



Increasing faculty familiarity 
with system. 

Increasing student familiarity 
with system. 

Improved availability of system 
(or increased capacity) 

Instructor’s interest or incentives 

Encouraging additional time for 
software development 

Adequate "off-the-shelf" materials 
(software) 

Consistent and adequate budgeting 



Adequate supplies and materials 
(tape, film, etc. ) 



^Faculty Training 



It is often useful to assess faculty attitudes 
gerning the increasing or decreasing use of mediated 
ching systems in colleges). 



D you feel that the quality of your overall 
struction is enhanced by the sometime use of 
ulti-media systems? (i. e. : does effort to 

ivelop good mediated instruction "rub-off" on 
)ur regular instruction pr epara.tions or acti vi- 
es ? ) 





lO 




es use of multi-media allow a better utilization 
teaclier resources ?• 

How so? 



ith respect to time use when teaching with 
ediated instructional systems, do you find 



at; 



you "save" time, how do you devote the 
n6 you lictv6 saved? 



you "require more time" where do you 
to obtain time for mediated instruction? 




I I Use more time overall 
Use same time overall 
j — I Save time overall 






□ further preparation 
j j unore group^ instructionj 
n other teaching q tutoring & counseling 
activities q evaluation activities 



Q Off campus consulting 
& conferencing 

□ Re search k writing 

□ Admi nistrative dhties 




n preparation 
□ instruction 

□ Teaching duties Q tutorial 

I I evaluation 
\ I Off-campus consult/ confer etc 

□ Research h writing 
[~) Administrative duties 

□ Leisure- owntime 

□ Other (specify) 





These questions seek information on the relation "between 
use -of media and quality of instruction generally.' 



EFFECT ON INSTRUCTION 
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\Tould additional opportunities for 
raining in the use of teaching systems be 
,n important factor in increased media 
iystem use? 



rhich of the following elements of faculty 
rientation or training would be of most 
alue to you? (Rate according to the 
reviously defined rating system) 



L your view, ' is the use of the system in- 
reasing;, stabilized, or decreasing at this 
oint in the life of the teaching system? 



□ Yes 

□ No 













Mechanical or electronic mani- 
pulations of the equipment 

Techniques of preparation of 
software for the system 

Improvement of presentation and 
teaching techniques (personal 
presentation shills) 

Technical assistance and orienta- 
tion in the preparation of teaching 
objectives, course development, 
etc. 

Educational Psychology 
■ Overview of media 
. Learning theory 
Others (please specify) 




Steeply 








Slowly 




Flat 



Decreasing 



ERIC 
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FORECAST 



Effectiveness of a system can sometimes 
be inferred from people’s intention 
to expand its use or size. 



Looking forward one year: 
Will the intensity of use be: 



1 J 1 f 1 J 




1 smaller same half doubled more 




than 


times 1 


doubled 


larger | 

■Tr-r* 



Will the size of the system be; 




smaller 



same’ half doubled more 
again than 

larger 



doubled 



times 

large] 




expansion of this type of system: 
During next one year budget? 

During next five-year period? 




(round figures) 




o 
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ometimes there is one faculty member of pervasive influence 
hose energy ’’povrers" the multi-media system. The control- 
ing factor here is whether the removal of such a person by 

dyancement or employment in some other institution v/ould C!J I R 

ave a gross adverse, effect on effective multi-media system LEADERSHIP 

mplementation. 



Do you feel there is one person whose absence 
would cause the program to become static or 
lose momentum? 

If yes, give title.' 






1 



□ Y 



es 



□ No 



Title 



Jetting aside, for the moment, questions of general 
jontent appropriateness let us examine the signal quality , 
[•idelity, ^'presence” and ’’style" factors in teaching 
aaterlals . 

Generally speaking, is the teaching material 
used in your system of a sufficient fidelity or 
teclinical quality to take advantage of the opera- 
tional qualities of the hardware in your system? 



Are you satisfied with the present status of the 
software use in the system? 

If no, what do you think can be done to 
improve the situation? 



FIDELITY REQUIREMENTS 



n Software loses more fidelity and 
quality through mediation than it 
should, 

□ Software quality is consistent with 
mediation system quality . i 

I I Software of low quality is often used ; 
in a higher quality mediation system. 







□ Yes 


□ No 












k 



h 



Have functional specifications been written 
(or adopted) for the software used with this 
system ? 



Please supply and insert behind this page a 
system diagram or sketch showing how the 
components of the system are related or used 
(in the functional relationship sense). 



In general, are the functional requirements 
of the teaching task met by the operational 
characteristics of the system? 





Che selection of "off-the-shelf" software for use in college 
teaching is often left to the individual teacher who is 
preparing the course. Other measures and efforts can add 

to the effectiveness of selection procedures. SOFTWARE ^ELECTION 
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allowing is ci list of means for mailing 
Igements concerning the effectiveness of 
.idling materials. Please ra.te (use 
eviously defined rating system) those 
3 ans now used to guide selection of teaching 
iterials: 



— » Intuitive judgement of responsible 
professors 

— Collective intuitive judgement of respon- 
sible team, "jury” or committee 

— Effectiveness rating based on pre-test - 
use-post-test study 

•^“^Student preferences or ratings from 
earlier course presentation 

-—'Materials selections based in part on 
state, regional or national course 
development activity (give source (Modern 

— Language Assn. , medical groups, 
engineering groups, etc. 

— Other (please specify) 





f you use means other than those listed 
ibove for estimating or establishing the 
effectiveness of teaching materials prior 
;o their adoption in the classroom or teaching 
system under evaluation, please describe: 




It Is sometimes useful for faculty to analyze the 
tlve cost of preparing unique materials, using 
entional teaching techniques, and using prepared, 
based materials. The following questions may help 
at this question. 



SOFTWARE COST 



Fake into consideration the cost of: pub- 

lished materials, texts, library resources 
tfvhich are allocated for this course (assigned 
3ind research), and other information re~ 
sources which are purchased from outside 
sources. Attempt to assign a cost per student 
hour for purchased material/ Consider life 
expectancy of text materials, life expectance 
of library acquisitions and other factors 
which cause cost per student hour to rise. 



Computation 

Gross cost of all 
published materials 



used in course 



Gross number of 
hours student time 
spent with published 
materials 



Cost per hour 
of educational 
activity derived 
from published 
materials 







1 

i 
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1 



J 



t 



[Please perform a similar computation 
[for prepared materials. Here again 
[effort should be devoted to estimating 
life expectancy of on-campus prepared 
materials. Are lecture notes, tapes, lab 
guide worksheets, and other locally pre- 
pared materials used for more than one 
semester? How does this effect the cost 
per hour of instruction estimated? 

Computation 






cost of staff time 



number of student 
hours 



Cost 

per 

hour 






Gross dollar cost 
of staff time spent 
preparing for and 
delivering lectures, 
and locally prepar- 
ing teaching materials 



Gross number of stu- 
dent hours using 



Cost per hour 
of educational 
activity derived 
from on-campus 
prepared materials 



locally prepared 
materials and attend- 
ing lectures 
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CRITIQUE 



We would liXe to have some information from you concerning the method 
and content- of our study. 



Are there questions you expected us to ask which we have overlooked? 



I^lease discuss.^ 




What do you like about this project? 



What do you not like about this project?^ 



o 





!d Instrument for the Evaluation of Learning Devices 

NDUCTED BY THE AMERICAN INSTITUTES FOR RESEARCH 



RESPONDENT 

IDENTIFICATION: 



t 

Name 



NSTI TUTiON 

, j. , , , , - I - -I I - - I — ■ T — -- « ^ - PI * 




(At Large) 



This questionnaire is part of a larger study 
conducted by the American Institutes for Research in 
which Faculty, Staff, and Students are being asked to help 
evaluate some of the Multi-Media Teaching Devices in use 
on this campus. Your responses will be merged with the 
responses of others; your ideas and reactions will be 
treated confidentially; and no personal identifications 
will be included in reports concerning the study. 



o 




Ii general, how well are you acquainted with 
he media teaching-learning system 
tsed on this campus? 



What is your attitude toward the media 
system on this cam pus ? 



What is your attitude toward the use of 
these newer methods in higher education^ 
in general. 



What do you think about the cost of these 
systems used 



I I Never heard of it 
□ Am familiar with it 

I I Have used it 
Q Other (designate) 



=1 : 






□ 


Positive 




□ 


Neutral 




□ 


Negative 




□ 

• 


Other (explain) 










_ . . — __ — 1 ■■ ■ ■ I i-Mi Tfc-mffTiTri>Mwimwn 


□ 


Positive 




□ 


Neutral 




□ 


Negative 




□ 


Other (explain) 





□ Don't know 

□ C ost too much 

□ c ost is commensurate with 
benefits 

I I Benefits outweigh the costs 
O Other (specify) 




1 
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What do you think about the cost of these new 
systems, in gene r_ajj? 



Do you think the present use of the system 
on this campus represents a saving in 
student learning time? 



[m Do not know 

CD The costs tend to outweigh 
the benefits 

I 1 The costs tend to be com- 
mensurate with the benefits 

CD The benefits tend to outweigh 




the costs 

CD The cost-benefit comparisons 
would vary with the type and 
use of a particular system 

CD Other (please specify) 



□ Yes 

□ Do not know 

□ No 

□ Other (please explain) 




Do you think that the effective use of such 
systems can save learning time for students? 



□ Yes 

CD Do not know 

□ No 

□ Other (please specify) 



Do you think that the goals of the multi-media 
system being used on this campus are presently 
being achieved? 



□ Yes 

CD Do not know 

□ No 

CD other (please specify) 



o 




Do you think the goals of this institution would be 
better served by appropriate and increased use of 
such multi-media systems? 






□ Yes 

- D Do not know 

□ No 

□ Other (please specify) I 








Do you think that the use of such multi-media 
systems can lead to more effective use of the 
instructors' time? 



Please use the following space for additional 
comments or questions ? 




3 . 
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CRITIQUE 



We would like to have some iufoi-matioii Irom you conccrmng the method 
a.iicl content of our study. 



Are there questions you expected us to cisk which we ha\ e o\cil.ooked. 
Please discuss. — — 



What do you like about this project? 



What do YOU not like about this project? 

^ • 
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I Instruiiieni for Iho Evaluation of Leas ning Devices 

jpUCTED BY THE AMERICAN INSTITaS;S;C FOR kSSSARCH 



RESPONDENT 

IDENTIFiCATION; 
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LEARNING DEVICE 



DENilF! CATION; 



1 ns ti tut ion 



□ CAI 



ED D AIRS 



□ ETV 

□ LL 



D 1 



iT^ 

Kj 



BASSE M □ MICE OTEA CHING 



1 



This questionnaire is part of a larger study 
cond’cicted hy the Arxierican Insr.il. Lite b for Res Starch in 
which Faculty, Staff, and Students are being asked to help 
evaluate some of the M.ulti- Media Teaching Devices in use 
on this C3.rnpus. Yo'o.r responses will be merged with tne 
responses of.others; your ideas and reactions will be 
treated confidentially; and no personal identi.fications 
will be included in reports concerning the study. 












TECHNICIAN 



INDEX 



Identification of System . . 1 

Response Options ^ 

Student Control of Teaching System Advance . .4 
Feedback Capa cili ties .... .o ^ ^ 

Rationale for Selection . . . . = . ^ 

Start-Up Phase . . . » .9 

System Capacity. . 12 

Reliability 13 

Fidelity Requirenaents 14 

Critique 19 
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IDENTIFICATION OF SYSTEM 



What is the name of the system? 



Do you consider this a discrete system 
or a sub- system electronically 
interconnected with other electro- 
mechanical teaching complexes or 
systems ? 

On the next page please check all the 
input means used in connection with this 
system 





' ' 


• 








* 1 

□ Discrete □ Interconnected 



o 

ERIC 



INPUT ARRAY 



=T'ilm 



p’P r inte d 



?«^3W5WfSvW' V erba.1 



^ IP C 1* »»<’^^********'^'*'^'^^^-j 



«’^Oxide 



TMicrofichc 


□ 


ie:ERIC 


Strip film 


□ 


MAST 


'v Transparencies 
^Slides (2x2) 


□ 

□ 


Print 


(lantern) 


□ 


only 


1 Books 


□ 


Texts 


1 Worksheets 


□ 




4-Panel/ charts 


□ 


W allmaps 


1 Games 


□ 


Monopoly 


5 Rolls 


□ 


"printout' 


“iys-Braille 


□ 




®4Tape 

-pises 


□ 


audio 


□ 


• computer 



Flat stock 
Stripe on film 



. fOn film edge 

-Spoken . 

^ film fr 



ames 



i^T./r •( • 1 fRecords' 

’"Mechanical 

I'Realia Live (lectures) 









Manual AlplmbcjJ;_. 

Film 






■Still 






[Strips 
Slides 
Books 

■••’Pape r Loose sheets 



□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 



Pictorial 



LDX-FAX, etc. □ 
Cathode ray -Tonotron/VTR Q 
I '‘Still"/Westinghouse disc. [[]] 



, ?»i.Motion 



ealia 

r^Film 

'Tape 

■TV 

^d-zive 



^Objects 
IwKits 

t^l 6 mm/ 3 5 mm 
,8mm loop 
y VTR 
4y y»%.^ ’s»c=3c-v7tu^»r;> Live 

^^Demonstration 
iRole play 8c 
. Mother student 









□ 

□ 

□ 

□ 

□ 

□ 

□ 



memory 

l6mm sound 

on film eqpt. 
Educ. radio 
Regular lo 

Kalart device 

331/3/45/78 

Dictaphone 

Profe s sors 

Sign Language 



Texts, 

Pamphlets 

LDX 

’’Stop Action" 



To Be 
As sembled 
Conventional/ 
Silent 

Technicolor 

"Instant 

Replay" 



Vdd categories if the system has other 
neans for presenting information (or stimuli). 
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a-ho i*o:iponr.es evoked by mulU-media toachiiis systems 
try over a v;ido ran^e. Some are in the: form of 
louchts, insxchts and understandincs which are never 
eprcnr.ed externally. Others, like page-turninc in a 
:*ov/derlan branched program, are only the cxtci'nal 
ascrvable evidence of far more compl'i cated responses 
ad chains of reasoning. The following questions 
btempt to c3arify and define the response options 
ossible with the media system under study. 



RESPONSE OPTIONS 



heck all of the response modes characteris. 
tically used by students in courses v^^bich 
use this media system. 



unie r i c al respon.ses 

D Handwritten computations • 

D Key- punched computations 
] Marking selected answers 
3 F.r>inHn^g__nume rals as symbols 






Isual response patterns 
3 Drawing sketches 

3 Selecting displayed pictorial alternatives 
3 Light-pen constructions for electronic 
comparison to models 
3 Performance responses 
□ Constructions 
I I Completions 
I I Generalized role-play 
|~3 Apprentice emulation 



Ian you define other response modes 
diich students use with this system? 
‘lease enumerate. 
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V erbal Response 

□ No, overt action 

[31 Push buttons (Edex) 

Q Page-turning (Crowder) 

□ Ma rk sensing (Tests) 

[3 Symbol Entry (Entry of a symbol in 
a blank space to denote choice of 
alternatives . 



Utterances 



f~3 Unrecorded 
Recorded 

□ For immediate comparison by monitor 
I I For immediate comparison by student 

I I For later playback by monitor 

□ For later playback by student 



Hand written 




I I Unsystematic notes, not saved 
[3 Preserved for student use and evaluation 
[31 Preserved for monitor comparison or 
correction (papers, etc. ) 

I \ Displayed on cathode ray tubes 



Typed verbal responses 

□ Unpr e served 
[D Preserved for student comparison (notes) 

I I Preserved for monitor evaluation (papers, 
reports, etc.) 

□ Typed into computer for comparison with 

computer -held model. 
n Held in computer memory 
r~l L ost after comparison 



1 



3 



Arc there other response options 
not mentioned above or sub- 
classifications not given above 
which would prove more meaningful 
to educators attempting to plan 
educational presentations? If yes, 
please mention in the space provided. 




STUDENT CONTROL OF TEACH ING 
SYSTEM ADVANCE 



Does this system permit student 
responses or .other activities to 
control the development of the 
instructional presentation? 

If yes, which of the following types 
of response can be accomodated? 




I I Constructed responses (utterances, 
■ typewritten entries, etc. ) 

IZ) Selected responses (push-button, 
light-pen, etc. ) 

Q Other (please describe) 
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Feedback of knov/ledce of the correctness of 
his I'osponse to tlio student is frequently an 
important part of a teachins system.. The correct- 
ness of these feedback provisions and their pi-ompt- 
ncss have a direct bearing on the efficiency of 
the teaching system. The following questions 
pui’suo this issue. 



FEEDBACK CAPABILITIES 



Docs this teaching system make 
provision for any feedback 
concerning the correctness of 
student responses during the 
periods of use? 

If yes, what is the approximate 
delay between emission of a 
response and the provision of 
feedback (give time in seconds 
or fractional parts of a second)? 





Which of the following statements 
best describes the nature of the 
feedback which is provided? 



I I A correct model for self- 
comparison by the student 

i I A simple announcement of the 

incorrectness or correctness 
of the response (comparison 
with the model having been 
accomplished by the system 
equipment rather than the 
teacher or student). 

□ Cor rective feedback (equipment 
reports on deficiencies in the 
student response and provides 
corrected response). 

Q Other (please specify) ______ 
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Is ihiB system an "all or nothing" system 
(that is, do system failures block out all 
part of the system)? Or docs failure 
cause a progressive attrition in 
signal quality or learning facility Vv^ith- 
out shutting down the systejii completely? 

Total system failures: 



Modules or units fail: 
Quality deteriorates: 



or 



1 □ All 


f~l Part 1 


[□ Possible 


1 1 ComiTLon 1 


1 P] Unlikely 


1 1 Never 


1 P) Often 

! 1 jNevc r 


I 1 Sometimes 


□ Often 


1 1 Occasionally 


O Seldom 




CIl Never 



Are you satisfied with the present 
functioning of the hardware in 
the system? 

If no, what do you think cotxld be 
done to improve the situation? 




Describe 



RATIONALE FOR SELECTiON 

I ecnoral, vrhen essentially similar 
'.ulus material could have been conveyed 
)thor means at lovfer cost, it is important 
recapitulate the ai’girnients used to Justify 
selection in the first place . 



Why was this system selected over 
other systems capable of conveying 
equivalent signals to the same 
sensory apparatus and decoding 
paths? 




o 
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(1) If system is CAI, why were 

concepts like PI, Text and test, 
and workbooks passed over? 



(2) If system is DAIRS, why 
were language lab, phonograph, 
telelecture, live -speech and 
radio, etc. passed over? 








(3) If system is Multi-Media 
classroom, why were carrel 
applications , audio -tutorial 
methods, conventional audio- 
visual programs and library 
resources passed over? 





(4) If system is closed circuit 
television, why were open -circuit 
television film programs, tape/ 

slide programs, phonograph-plus - 
filmstrip presentations and other 
^mailable multi-sensory media passed 
over ? 




(5) If system is video-tape 
recorder, why were role play, 
games, classroom simulator, 
etc. passed over? 
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Still anotlicr question: Arc any of 

the following reasons important in 
understanding the selection of this 
specific system over others of its 
class? 



I I Improved teaching of regular subjects, 

( I Standardized teaching of regular 
subjects. 

I ! Pedagogical innovation made possible. 

□ imp roved fidelity of image made 
possible. 

I I Equipment reliability imi^rovement 
expected. 

i I 'Economy anticipated (explain how) 






Q Versatility of medium compared 
to other alternatives. 

{ i Acceptability of this medium to 
teachers. 

I I Acceptability of this medium 
to students. 

I I Motivational aspects of student 
response to equipment. 

I I Other reasons (please explain) 



o 

ERIC 



r 4 
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Panel A 
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START-UP PHASE 



The preceeding diagram labeled, "panel A", indicates 
at a number of preliminary activities can be grouped 
gethor and referred to as the "start-up" period, 
osely speaking, these are all the events v;hich occur 
fore faculty are permitted to schedule regular, continuing 
“line academic use of the system. 



What kinds of problems were 
encountered during the "start- 
up" phase? 



During the design phase were 
functional specifications written? 

If so, please provide; if not, please 
re-.con struct in the space provided. 




I I tune-up and shake -down 



□ manufacturer doesn't follow through! 

□ . component not compatible or I 

appropriate 

I I failure of delivery schedules 

□ other (specify) I 






Were technical specifications written 
(clescrihing the mechanical and electronic 
operations and equipment which would 
be necessary to achieve the functions 
described in the functional specifications)? 

If so, please provide; if not, please 
re -construct in the space provided. 



Have functional and technical 
specifications been written (or 
adopted) for the software to be 
used with this system? 



Please supply a system diagram or 
sketch showing how the components 
of the system are related or used 
(in the functional relationship sense). 

Use facing page. 




Are there special ar rangeirieiits and 
hook-ups which are possible in this 
system and provide unusual functions 
or multiply the capacity of the system? 




1 0 
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In general, arc the functional 
requirements of the teaching 
task met by the operational 
characteristics of the system? 

If no, please specify problems. 



The follov;ing series of questions attempts to 
determine the capacity of the system measured in units 
of student use. (Disregard actual use and concentrate 
instead on optimum use of the system.) 



What is the total number of 
student positions? 

How many houi’s each week is 

the system available for scheduling 

students? 



How many hours should be deducted 
for unpredicted system failures? 



How many hours should be deducted 
for scheduling problems ? (passing 
classes, mealtimes, conilicts, etc.) 



How many hours should be deducted 
for normal maintenance activities? 

How many houi’s should be deducted 
for system loading (set-up time, 
previewing, etc, )? 
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System Capacity 







Unpredicted failures 
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What is the total of deductions? 



Total deductions 



After making deductions what is 
the remaining available time? 



Remaining time 






MultiiDly this last response by the 
number of student positions (See page 16)' 
to obtain the gross number of 
student hours available under 
optimum conditions. 



Student hours (optimum) 



In your view, is the use of the 
system increasing, stabilized, 
or decreasing at this point in the 
life of the teaching system? 




This series of questions has to do with 
reliability. It should help you detect 
problems v/hich are limiting the effective 
use of your instructional system by in- 
terrupting or degrading the system perform- 
ance . 



RELIABILITY 



Is system failure or image quality 
degradation ever a problem with 
this system? 



I } Yes --continue questions. 

O No -- skip to next major subject 
heading. 



Do failures of this system cause 
total or partial loss of the educational 

effect of the system? D Can fail in part 

i I Can fail in total 
n Quality deteriorates 
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Instrun'iGnt for tho Evcilucuion oi Lssrnin9 DgvIvG^ 

®UCTED BY THE AMEIUCAN INSTITUTES i'OR RESEARCH 



RESPONDENT 

!DENTlFiCAT10N: 



Name 

Title 




O 






J 




if 

J 



\ 



a 
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LEARNING DEVICE 
IDENTlFlCATiON: 




This questionnaire is part of a larger study 
conducted by the American Institutes for Research in 
which Faculty, Staff, and Students are being asked to help 
evaluate some of the Multi-Media Teaching Devices in use 
on this campus. Your responses will be merged with the 
responses of others; your ideas and reactions will be 
treated confidentially; and no personal identifications 
will be included in reports concerning the study. 
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This section is designed to collect historical 
data on the development of phases through which 
the system progressed in coming to its present 
state. 



Who was the originator of the idea 
for this teaching system? 




Was there a problem or difficult 
situation that generated a need 
for which this "system" was offered 
as a solution? 

If so, please state. 



Was this system origina'^ed from a 
need to "keep up with the Joneses" 
emanating from administratior s or 
development office personnel? 





Was this system considered 
innovative or original in its 
conception and early development? 

If so, how? 




o 




Have the goals or objectives been 
!'• specified at any time? 

If yes, when? Please describe. 

Are copies of goals or objectives 
available. Please affix, or 
list on facing page. 



Did the original "seed money" 
come from: 



Can you trace the developmental 
stages through which the system 
has gone? 

If yes, please specify on facing 
page. 



What is the name of the system? 



Do you consider this a discrete system 
or a sub“ system interconnected 
with other electro ^mechanical teaching 
complexes or systems? 

On the next page please check all the 
input means used in connection with this 
system 




Yes CH No 





n Discrete □interconnected 

< 







2 



INPUT ap:ray 






|=Film 



>i?2ica5>Verbal 




l^’Printed* 



laPaper 



I^Ernbo 



ssed*^ 



fOxide 



Radio 
rSpoken Optical 




J Microfiche Q 

Strip film Q 

Transparencies Q 
-^Slides (2x2) Q 
(lantern) Q 
□ 
n 
□ 
□ 
□ 
□ 
□ 
□ 
□ 
□ 



Books 
Worksheets 
=Panel/ charts 
Games 
Rolls 
=^Braille 
fTape 
■Discs 
Flat stock 
Stripe on film 



^‘Mechanical 




On film edge 
Jfi film frames 
ecords 
elts 



la®sEi£=ag> P i c t o r i a 1 




3s,Manual Alp 



Still 




Realia Live (lectures) 

“*n^trips 



Film 



Paper- 



fsiides 
Books 

Loose sheets 
LDX-FAX 



etc, 



;:®Motion 




□ 
□ 
□ 
□ 
□ 
□ 
□ 
□ 
□ 
□ 
□ 

athode ray-Tonotron/VTR [H 

"Still"/Westinghouise disc« [j 
Objects 
its 

6mm/ 3 5mm 
Smm loop 
VTR 
Live 

f^Demonstration Q 

E ole play & Q 

ther student 



ealia 



Film 

=Tape 

TV 



^Live — 




□ 

□ 

□ 

□ 

□ 

□ 



Add categories if the system has other 
means for presenting information (or stimuli) o 



o 
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MAST 

Print 

only 

Texts 

Wallmaps 

Monopoly 

"printout" 

audio 

computer 

memory 

l6mm sound 

on film eqpt. 
Educ. radio 
Regular lo 

Kalart device 

331/3/45/78 

Dictaphone 

Professors 

Sign Language ; 

i'. 

\. 

p 

Texts, 

Pamphlets f 

LDX i 

"Stop Action" * 

To Be I 

Assembled \ 

Conventional/ [ 

Silent I 

Technicolor j 

"Instant I 

B e play ' ' i 
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5he responses evoked by multi^media teaching systems 
over a wide range. Some are in the form of 
Slights, insights and understandings which are never 
sressed externally. Others, like page-turning in a 
3Wderian branched program, are only the external 
servable evidence of far more complicated responses 
a chains of reasoning. The following questions 
tempt to clarify and define the response options 
ssible with the media system under study. 



^eck all of the response modes characteris 
ically used by students in courses which 
Lse this media system. 



imerical responses 

1 Handwritten computations 
1 Key- punched computations 
1 Marking selected answers 
] Entering numerals as symbols 



Lsual response patterns 

! Drawing sketches 

] Selecting displayed pictorial alternatives| 
] Light-pen constructions for electronic 
comparison to models 
] Performance responses 
r~i Constructions 

□ Completions 

□ Generalized role-play 

i \ Apprentice emulation 



you define other response modes 
:h students use with this system? 
j,se enumerate. 



al Response 

I I No overt action 

□ Push buttons (Edex) 

□ Page-turning^ (Crowder) 

[3 Mark sensing (Tests) 

□ Symbol Entry (Entry of a symbol in 
a blank space to denote choice of 

alternatives. 




Utterances 

r~l Unrecorded 
Recorded 

n For immediate comparison by monitor] 

□ For immediate comparison by student 
For later playback by monitor 

□ For later pla'yback by student 

g.J ■ I I - I «I I ' ' I I *■ ■ - I I ' I III""' ' 

Hand written 



□ Unsystematic notes, not saved 
Preserved for student use and evaluation 

□ Preserved for monitor comparison or 

L correction (papers, etc. ) 

I I Displayed on cathode ray tubes 

Typed ^rbal responses 

n Unpreserved 

□ Preserved for student comparison (notes) 
Preserved for monitor evaluation (papers 
reports, etc.) 

i I Typed int: computer for comparison with 
computer-held model. 

□ Held in computer memory 

Lost after comparison _============» 
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Are there other response options 
not mentioned above or sub- 
classifications not given above 
which would prove more meaningful 
to educatiors attempting to plan 
educational presentations? If yes, 
please mention in the space provided. 




STUDENT CONTROL OF TEACHING 
SYSTEM ADVANCE 



Does this system permit student 

responses or other activities to 
con^^’rol the development of the 
instructional presentation? 

If yes, which of the following types 
of response can be accomodated? 






Feedback of knov/ledge of the correctness of 
his response to the student is frequently an 
important part of a teaching system. The correct^ 
ness of these feedback provisions and their prompt- 
ness have a direct bearing on the efficiency of 
the teaching system. The following questions 
piiTsue this issue. 





Does this teaching system make 
provision for any feedback 
concerning the correctness of 
student responses during the 
periods of use? 

If yes, what is the approximate 
delay between emission of a 
response and the provision of 
feedback (give time in seconds 
or fractional parts of a secona)? 



Which of the following statements 
best describes the nature of the 
feedback which is provided? 






I I A correct model for self- 
comparison by the student 

r~i A simple announcement of the 

incorrectness or correctness 
of the response (comparison 
with the model having been 
accomplished by the system 
equipment rather than the 
teacher or student). 

□ Corrective feedback (equipment 
reports on deficiencies in the 
student response and provides 
corrected response). 

D Other (please specify) _____ 
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Is this system an "all or nothing" system 
(that is, do system failures block out all or 
part of the system)? Or does failure 
cause a progressive attrition in 
signal quality or learning facility with- 
out shutting down the system completely? 

Total system failures: 



Modules or units fail; 
Quality deteriorates: 



Are you satisfied with the present 
functioning of the hardware in 
the system? 

If no, what do you think could be 
done to improve the situation? 



fn All 


1 1 Part 1 


1 CH Possible 


1 1 Common 1 


1 1 Unlikely 


i 1 N"e ve r | 


□ Often 

riNever 


(~1 Sometimes 


1 1 Often 


1 1 Occasionally 


n Seldom 

- - - — 


r~l Never | 


It ■ ■ IT II ~l ‘ " ' ■ ■ II ■ II ^ 






genera.!, when essentially similar 
ul\is material could have been conveyed 
ther means at lower cost, it is important 
ecapitulate the arguments used to justify 
selection in the first place. 



A TH AM A . . 

n I iui\!nL! 



Why was this system selected over 
other systems capable of conveying 
equivalent signals to the same 
sensory apparatus and decoding 
paths? 





(1) If system is CAI, why were 
concepts like PI, Text and test, 
and workbooks passed over? 



(2) If system is DAIRS, why 
were language lab, phonograph, 
telelecture, live -speech and 
radio, etc. passed over? 



(3) If system is Multi-Media 
classroom, why were carrel 
applications, audio -tutorial 
methods, conventional audio- 
visual programs and library 
resources passed over? 



(4) If system is closed circuit 
television, why were .open-circuit 
television film programs, tape/ 

slide programs, phonograph-plus- 
filmstrip presentations and other 
mailable multi-sensory media passed 

over ? 



(5) If system is video-tape 
recorder, v/hy were role play, 
games, classroom simulator, 
etc. passed over? 



Still another question: Are any of 

the following reasons important in 
understanding the selection of this 
specific system over others of its 
class? 



CU Improved teaching of regular subjects. 

□ Standardized teaching of regular 
subjects. 

n Pedag ogical innovation made possible. 

0 Improved fidelity of image made 
possible. 

1 I Equipment reliability improvement 
expected. 

□ e conomy anticipated (explain how) 



□ Versatility of medium compared 

to other alternatives. 

□ a cceptability of this medium to 

teachers. 

I I Acceptability of this medium 
to students. 

I 1 Motivational aspects of student 
response to equipment. 

n Other reasons (please explain) 
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Estimate the actual gross number of hours | 
of student use per annum for this system. | 

(Compute average positions used x hours I 

scheduled per week x weeks used per year t) I ( ) x f ) x ( 

I positions ■ hours/wk wks/yr 

= C ) 



Is the service "adjunctive 
support" or "whole -course - 
presentation" . 




gross hours use 





Is this system availa.ble to the 
students on demand (first come, 
first served) or do they have to 
reserve or "sign-up" for each 
use of the system? 
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COST (HARDWARE) 



Use actual or estimated figures in 
giving the following costs; In 
computing original hardware cost 
of the system, the following factors 
should be considered: 






Are there other elements of original 
cost which are hard to determine? 
What are these cost categories? Any 
estimates available? 




ji 



$ consultants hired to write 

• specifications. || 

i) 

visits made to installations i 

for inspections. |j 

Ij 

costs of ordering, receiving i 

shipping, storage, etc. j 

cost of the space in which | 

the equipment stands | 

j 

costs of remodeling or ^ 

rehabilitating the space to j | 

meet the system needs. | | 

cost of hardware, wiring, and j 

other direct system components | 
(including software if sold as | 

part of the hardware package). | 

cost of installation of system. 

cost of rework, modification, 
and extras in order to make 
the system work. 

cost of training personnel, 
other costs. 



j 

I 



I 




In this section, the maintenance 
and replacement costs are to be 
accumulated. Estimate the cost 
pe r annum of the following: 



I ^ Cost of specialists or technicians 

• to diagnose or make repairs 

Cost of outside repairmen coming 
in to assist technicians 

Cost of technicians wages and 

# 

equipment 

Cost of replacement parts and 
modules . 

Cost of heat, light, and janitorial 
se rvice s 

ost indicating a share of the | 

university administrative budget 





Are there other continuing costs 
v/hich should be considered as 
part of the maintenance budget? 



At the time of purchase what expected 
life was anticipated for equipment? 

overall years of use 

or: estimated # of student hours 

of use 



After m-use experience , what is 
the present estimated actual life 
of equipment? 

overall years of use 

or: student hours of use 



Anticipated and actual life of equipment are frequently measured in different 
terms. Actual life is given in student use hours, whereas anticipated given 
in calendar years. Check to see that both of the foregoing figures are given 
in the same units. 



years 

student hours 




□ Yes _ □ No 

Amount 
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What is your estimate (in dollars) of 
the cost per hour per student of this 
system? 



$ 



per student hour 



Amortize original cost (page 11) over estimated life; (bottom, page 12); add annual 
cost (top. page 12); divide by estimated hours of student use (page 10). 



Please mention any factors (used to 
compute the "cost per student hour 
of use") which have not already been 
discussed. 




START-UP PHASE 



' The previous diagram labeled , "panel A .Indicate 
at a nLber of preliminary activities can be grouped 
4 -v,at. and referred to as the "start-up period. 

^aely speaking, ‘"ese are all oontlnulng 

.fore faculty are permxtted to schedule regu 

i-line academic use of tie system. 



I 



What kinds of problems were 
encountered during the "start- 
up" phase? 



□ tune-up and shake-down 

I [ manufacturer doesn't follow through 

□ coinponent not compatible or 

appropriate 

□ failure of delivery schedules 

□ other (specify) 



During the design phase were 
functional specifications written? 

(In pedagogical terms) 

If so, please provide; if not, please 
re-construct in the space provided. 
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Were technical specifications written 
(describing the mechanical and electronic 
operations and equipment which would 
be necessary to achieve the functions 
described in the functional specifications)? 

If so* please provide; if not, please 
re-construct in the space provided. 



Have functional and/or technical 
specifications been written, 
developed in memorandum form, 
or adopted for the software to be 
used with this system? 



Please supply a system diagram or 
sketch showing how the components 
of the system are related or used 
(in the functional relationship sense). 

Use facing page. 

Are there special arrangements and 
hook-ups which are possible in this 
system and provide unusual functions 
or multiply the capacity of the system? 
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In general, are the functional 
re<^uireiTients of the teaching 
task met by the opera-tional 
characteristics of the system? 

If no, please specify problems. 




The following series of questions attempts to SVStem CapaClty 

determine the capacity of the system measured in units J 

of student use. (Disregard actual use and concentrate 
instead on optimum use of the system.) 



What is the total number of 
student positions? 

How many hours each week is 

the system available for scheduling 

students? 

How many hours should be deducted 
for unpredicted system failures? 

How many hours should be deducted 

for scheduling problems ? (passing 
classes, mealtimes, conflicts, etc.) 

Hov 7 many hours should be deducted 
for normal maintenance activities? 

How many hours should be deducted 
for system loading (set-up time, 
previewing, etc. )? 




Scheduling problems 



I 

Maintenance activities 



System loading 
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What is the total of deductions? 



Total deductions 



After making deductions what is 

the remaining available time? Remaining time _ i , _ 



Multiply this las^ response by the 
number of student positions (See page 16) 
to obtain the gross number of 
student hours available under 

optimum conditions. ^Student hours (optimum) 



In your view, is the use of the 
system increasing, stabilized, 
or decreasing at this point in the 
life of the teaching system? 









1 ! Steeply 


r~l Slowly 


— — 







□ Flat n Decreasing 



This series of questions has to do v/ith 
reliability. It should help you detect 

problems which are limiting the effective REL lABlLITY 

use of yo' r instructional system by in- 
ferrupting or degrading the system perform- 
ance . 



Is system failure or image quality 
degradation ever a problem with 

this system? CH Yes --continue questions. 

9 

CD No -- skip to next major subject 
heading. 



Do failures of this system cause 
total or partial loss of the educational 

effect of the system? □ Can fail in part 

□ Can fail in total 
i I Quality deteriorates 



o 
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FORECAST 



Effectiveness of a system can sometimes 
be inferred from people’s intention 
to expand its use or size. 



Looking forward one year: 
Will the intensity of use be: 



, 1 t - i .. 1 




smaller same half doubled more 




than 


times 


doubled 


larger 



Will the size of the system be: 



smaller same half 



again 

larger 



doubled more 
than 



times 



doubled 



larger 



Looking forward five years: 



By how large a factor will 
your needs be increased: 
(over the present system) 



What is your expected percentage of increase, 
in enrollment over 5 years? 



Please estimate the financial outlay for 
expansion of this type of system: 



Luring next one year budget? 



$ 



(round figures) 



During next five-year period? 
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\ 


» 1 




1 


1 


II ! ■ . ' ■ — 

0 


1/2 1 


2 


5 


10 times 
larger 




N. 



: 
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Systems vary in the degree of fidelity or 
image quality they can deliver. These 
questions explore the educational needs 
Imposed on the system and the equipment 
capabilities available to meet those needs. 



Fidelity Requirements 



What are the general needs which 
this equipment is expected to 
serve? 



Is this system under-used? That is, can 
this system do more than it is 
regularly called upon to do? (i. e. : 
using a video channel for an essentially 
audio transmission or using a multi- media 
classroom as a lecture hall). 



□ ’’Let us reason together” (abstract, 

symbolic verbal interaction). 

□ "L me show you” (demonstrative, 

audible and visible iconic messages) 

Q ’’You try it” (complex behaviors, 
actual performances, etc). 




Consider each of the appropriate 
variables for a visual system. 



Is color or black and white required? 

Is motion or are still pictures required? 



□ Color U B & W 

I I Motion n Still 



Is this system stereoptical or 
monocular? 



I n Stereoptical I I Monocular 



What is the maximum visual angle? 
(screen width as it relates to focal 
distance) 



Visual angle 




o 

ERIC 




What are the limits of focal distance? 



What is the TV resolution? 



What is the optical image resolving power? 



What brightness measure of visual image 
is used? 



What degree of brightness can be 
achieved? 



What degree of brightness is acceptable? 



What degree of color fidelity is 
obtained? 



What is the frame rate (in frames 
per second) f<^r motion? 



What is the change speed (in seconds) 
for still picture change? 













er|c 



Consider each of the appropriate 
' variables for an audio system. 



Is this system stereophonic or 
monaural? 



What are the frequency response 
characteristics of the audio 
portion? 



Do you have a frequency response 
chart showing acceptable charac- 
teristics? (If yes, please include 
a copy) 



If unusual frequency characteristic 
is desirable, please indicate hov/ 
it is useful. 



What "signal-to-noise” ratio is 
acceptable? 

What "signal-to-noise" ratio is 
achieved? 








o 
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Consider each of th ^ appropriate 
variables for printed materials 
or print-out devices. 





Focal distance 



Interlinear spacing_ 



Visual angle subtended (line length 
as it relates to focal distance) 



Speed 

(for print-out and visual display 
devices, in words or symbols 
per unit of time) 
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How do you measure lighting at 
reading surface? What is an 
acceptable level? What is 
achieved in this system? 



I 



What is an acceptable l eve l? 



What is achieved in this system? 
Line length. 



What is the ambient 
light level at working 
surface? 
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In order to estimte the effect of ^wlreliable 
operation It ic often necessary to compare 
the optimum performance with actual perform- 
•ahce. The follovring series of questions will 
help you determine the proportional effect of 
interruptions caused by unreliable equipment. 



System failures are; 



What is the total number of students 
using (or intending to use) the system 
each week? Make no adjustments for 
cuts, down-time, cancellations, etc. 
(see page 10) 

What is the avera.ge number of hours 
'each student is expected to use the 
system each week? Make no 
adjustments for equipment failures, 
cuts, etc. 



Multiply the foregoing figures to 
estimate the gross number of 
student hours a week. 



Estimate the number of student^’ 
effected by outright system failure. 



IMPACT OF FAILURES 

i| 

'1 

- . >1 

~ : r ' 

D Negligible {skip to next page) j 

□ Significant I |j 





estimated student hours 
of use each week 



# of students effected by 

failures in average week 



Estimate the number of students 
effected by deterioration of the 
image below learning thresholds. 



# of students whose learning 
is effected by degraded 
image quality 
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Estimate the average number of 
hours (or minutes) of lost time 
experienced by the students. 



# average loss due to 
failures 

— I -■ I ■« 1-rn ^ 

# average loss due to 

deteriorated image 
quality 



Compute the number of student hours 
lost each week for each class of 
defect in system. 

students lost time failures 

# students lost time deterioration 

total (add) = 



Compute the percentage of instructibnal 
time lost due to reliability problems. 



Please check, in the following list of | 

factors, those which have been considered 

in the foregoing questions when computing I 

down-time percentages. □ Receptor failures 

(headphones, terminals, 
monitors, etc. ) 

□ Generator failures 

(projectors, cameras, tape 
decks, etc. ) 

I 

ED Time losses due to poor training 1 

of employees (mixups, poor 
preparation) 

ED Failures due to defective information j 
storage components (film, tape, etc.) 

ED Budgetary and maintenance- 

ED Other major cause (specify) 



o 
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COURSE MATERIAL CONSTRUCTION 



Is special' consideration given in 
the form of decreased teaching load 
for staff m„embers who prepare special 
materials for courses? 


□ Yes 


□ No 


• 

Arc there any other considerations 
given? If so, please specify. 


□ Yes 


□ No 



Select an example of teaching 
material (VTR presentation, 
programmed instruction unit, 
language tapes) developed on 
this campus for use on this 
device. Identify and describe 
it. 



Name 



Description 



Content 



Format 



Technician Specialties 



Wliat provision is made for the 
determination of copy right 
ownership at this institution 



Q Professor owns all rights 
n Professor owns only those 
•things done outside the office 
I i University owns all rights 
Q Non-professors (technical 

personnel, grad, students etc. ) 
have norights to materials 
they prepare 



Do copyright and ownership rules 
impede or enhance The flow of original 
soft ware from Teachers? 



I I impede 
□ enhance 

n other effect: (specify) 
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Keeping the above-described example in 
mind, please check those elements of 
the following check- list which were 
accomplished during the construction of 
‘ the teaching material. 



. 1, . Establish or define course segment objective ^ ^ — _ Q 

Z. Specify terminal behavior — — ^ D 

3. Determine criterion perforinance — - — — — - D 

4. Test construction:. "entry” test items Q 

’Pest construction: terminal "final" test items — _ f]]] 

6. Plan field or chain of instructional activities ! — — L] 

7. Select content (concepts, ideas, skills) for each event^^^ Q 

8. Choose medium or media of instruction for each event _ — Q 

9. Write verbal portions of presentation ^ : O 

10. Prepare "story-board" of pictorial portion . - _ — — — C 

11. * Record verbal portions (tapes, etc. ) :! ^ Q 

12. Prepare or reproduce visuals (photos, charts) — : Q 

13. Correlate audio and visual elements tD 

14. Test construction: ("quiz" on this unit) — ^ — : [U 

15. Define appropriate response of learners O 

16. Administer tests at entry level ^ — — D 

17. Conduct or monitor teaching/learning situation^ — [3 

18. Administer tests at progress point (quiz) □ 

19. Score Item Analyze tests ' . ^ □ 

20. Write individual "prescriptions" for slow or fast learners _____ ^ Q] 

21. Administer test at terminal point - □ 

22. Revise materials in terms of experience .. — — — □ 

23. Other tasks ^ — - ^ □ 



!i 

ii 
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Comments: 



Do you have guidelines, procedures book, or other 
material describing your materials development 
sequence? . . ' 







□ no 

□ Yes, 

Please attach. 




I 
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[t iri sometimes useful for faculty to analyze the 
tive cost of preparing unique materials, using 
cntional teaching techniques, and using prepared, 
haood materials. The following questions may help 
at this question. 



SOrnWARE COSTS 



L’ake into consideration the cost of: pub- 

ished iuaterials, texts, library resources 
Mch arc allocated for this course (assigned 
md research), and other information re- 
sources which are purchased from outside 
sources. Attempt to assign a cost per student 
[lour for purchased material/ Consider life 
expectancy of text materials, life expectance 
of library acquisitions and other factors 
which cause cost per student hour to rise. 



Computation 

Gross cost of all 
published materials 
used in course 

Gross number of 
hours student time 
spent with published 
materials 



Cost per hour 
.of educational 
activity derived 
from published 
materials 



Cost, 



student 
time spent. 



Cost 
per hour 






Ccise perform ci simila.r computs-tion 
r prcpo.rccl mcitcria.ls. Here ags-in 
fort' sfioi-ild be devoted to estiiiiciting 
:e expectancy of on- campus prepared 
aterials. Are lecture notes, tapes, lab 
tide worksheets, and other locally p. e- 
i,red materials used for more than one 
^mester ? How does this effect the cost 
51* hour of instruction estimated? 

omputation 







cost of staff time 






number of student 
hours 

raemt3t9Mm ^ 



Cost 

per 

hour^^ 



4 

Gross dollar cost 
of staff time spent 
preparing for and 
delivering lectures. 

and locally prepar- Cost per hour 

ing teaching materials of educational 

— — — = activity derived 

from on- campus 

Gross number of stu- prepared materials 

dent hours using 
locally prepared 
materials and attend- 
ing lectures 



o 
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CRITIQUE 



we would UUe to have son.e information from you concerning the method 
and content of our study. 



Are there questions you 
please discuss 



expected us to ask which we have overlooked? 




What do you like about this project? 



What do you not like about this project? 
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lid Instrument for the Evaluation of Learning Devices 

INDUCTED BY THE AMERICAN INSTITUTES FOR RESEARCH 



RESPONDENT 

IDENTIFICATION: 




LEARNING DEVICE 
IDENTIFICATION: 




i 



Institution 



□ CAI □ ETV I 

I 

□ DAIRS □ LL 

, D MM classrmD microteaching 

■ '■ — ■■■ ■ ■ > .. ■ ^ - j 

i 



This questionnaire is part of a larger study 
conducted by the American Institutes for Research in 
which Faculty, Staff, and Students are being asked to help 
evaluate some of the Multi-Media Teaching Devices in use 
on this campus. Your responses will be merged with the 
responses of others; your ideas and reactions will be 
treated confidentially; and no personal identifications 
will be included in reports concerning the study. 
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INSTITUTIONAL CHARACTER 



IfJeW lnstru.«ent for the Evaluation of Learning Devices 
!L!>) contains several segments. This material 
, the identity and character of the Institution in which 
bi-media systems are being studjed. 

5 segment, conventional identifying information (such 
the type and size.of the student body of the institution) ^ 
i be collected. In addition, some more subjective qualitieo 
1 be discussed which have bearing on the objectives and 
Is which the multi-media system is called upon to support. 

> composition of the student body frequently yields impor- 
it information concerning the nature of the service the 
.lege 'is providing to its students. 

ease select one statement which comes closest to 
entifying the student group served at this institution. 



STUDENT BODY COMPOS ITION 



□ The majority are 

between 17 and 22 years of age 

□ There is a substantial portion 
of working persons and 
•’retreads" in an otherwise 
straight undergraduate student 
b ody . 

I I A very significant portion are 
mature j independent, working 
adults with a scattering of ages 
up into the 30’ s. 

□ Other thumbnail sketches 



Speaking generally , v/hat family income 
group tends to patronize this institution? 



□ Even mixture of wealthy, middle-class, 

and working class students. 

□ Predominantly wealthy and upper -middle 

class students 

□ Mostly middle-class youths whose parents 

are making a major contribution to their 
college costs o 

[U Predominantly working class with many 
students paying all or part of their ovm 

fees. I 

f~~| Other characteristization of the family 

income group. Specify 
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j propoi'l-lon of cominuterc vs. residents 
luontly helps establish the academic climate 
i collocc. 



COiVlIVlUTER/RESIDENT RATIO 



ease indicate the relative percentages of commuters 
d on- or near-campus residents. 



selection process often exerts profomd influences 
the character of an institution and on the educational 
grams and systems used in it. 

hat is the approximate College Entrance ExamJ. 
Ltion Board, ACT, or other test- cut-off 
:ore below which only a few unusual students 
heletes are accepted? 

re there regional, religious, social or cultural 
ctors which have a bearing on aspirations and 
»als of the young people, or which help to set 
e tone of the institution as a whole? Please 
jscribe in the spaces provided. 



'hat is the approximate geographic distri- 
ation of student origins ? 



% of commuters 



% of residents 



SELECTION PROCES 

I 



s 



Regional 



Religious^ 



Social 






Regional 
Within state_ 

Out of state_ 

Foreign 



CEEB scor 



•jlJ 

t«Mcaufa99Ks>assc 



ACT score 



%ile for other 
tests 



% 






% 



% 



% 



o the selection processes used here exert 
ny control or shaping forces on the: 



Content of your media system? 



or the Equipment mix of your MM system ? 
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is series of questions should be answered in terms of 
p oampus where the MM system is being studied. 

hat is the total student population on tM s campus? 



f the total how many are part-time and how 
i.any are full-time students ? 



■f the total what is the sex composition ? 

Can your institution be readily characterized? 

oes this college fall within one of the following 
ategories: (Mark two if needed) 



S^hat is the total population of the commuter 
tool area (or Metropolitan Area or Marketing 
Lrea) in which this college is located? 



s this college located in the: 



s this a single campus institution or is the 
nulti-media installation we are studying located 
m one of several campuses? 



normative data 




CHARACTERIZATION 




r~l Old land-grant university 
r~l Denominational college 
f ] Endowed private insitution 
n City -operated commuter college 
n two-year community college 
Q Younger burgeoning state 
university 

n Other (specify) 




total population 



I I Eastern metropolitan corridor 
I I Middle West 

□ Southwest 

□ Far West 
r~] South 



i I Single campus 
institution 

□ Multi -campus 
institution 



! 
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b all schools have the same mix of "years", 
lieges, sequences etc. 



grade levels served 




rhat is the estimated size of the following 
roiips ? 



freshmen 

sophomores 



juniors 

seniors 



graduates 



other (designate) 



\Thich of the statements below best describes or 
defines the organizational structure of your 
nstitution? 



Give the names of the major subdivisions 
»f this institution. 



I I Institution divided into 
schools 

M Institution divided into colleges 
CU Institution divided into 
, divisions 

n Other (designate) , 




5o you emphasize the four-year liberal art's program, 
he undergraduate specialization sequences, and 
he graduate level programs equally? 

If no, where is emphasis? 



:s this college or university: 



r 



□ Yei 



□ No 



I I Liberal Arts 
I I Specialization (engineering, 
business, etc.) 

□ Graduate programs 

FUNDING/CONTROL 



□ State supported 

or 

□ Privately endowed 

or 

□ Combination (explain) 



o 
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oes a board of directors or trustees help 
trrnulate school policy? 

How are they selected? 



’ The size and composition of the faculty often is 
esumed to have hearing on the character and success 
a school. 

ow many faculty members do you have? 



□ Ye, 



□ No 



I ii r t.— r f jt 



I I Election 
I I Appointment 
□ Self-perpetuating 
n Other 



FACULTY COMPOSITION 






(number) 



oncerning this total number of faculty members 
tease give rough, off the cuff, estimates of the 
>llowing breakdowns: 
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Intensity of service 



% part-time 



% f uU~time 



Sex composition 



_% male 
% female 



Degrees held 



_%Ph.D. or equiv. 
%M.A. " " 

% B.A. 



tl 



n 



Titles held 



_% hill professor 
% assoc, professor 
% asst, professor 
% instructor 



Salaries earned (put part-timers in 
the category they would be in if they 
were to work full-time) 

% under $5,000. 



_% $5000/7,500. 

% $7500/10,000 

% $10,000/12,500 
% $12,500/15,000 
% $ 15,000 & above 



FACULTY TIME USE 
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It Ir, frequently uceful to estimate the relatlonsh'p 
reen toachlnt' load and research time allotments. The 
jtlons are addressed to 

l:his institution faculty members use their 
e approximately as follows: 



the teaching time what proportions are 
oted to the major separate elements of 
teaching program (use fractions)? 



:i 

ii 




preparation 
group instruction 
tutorial & counseling 
evaluative activities 
other (specify) 



SALARY/RESEARCH/TEACH iNG CURVE 



Cow is a blank graph on which to 
icate the relationship which exists, 
your campus, between title and 
ary on the one hand and amount of 
iearch and teaching on the other. 



1 



full professor 
high salary 



asst, instructor 
low wages 



primarily primarily research 

teaching -- little teaching 

little research 




□ A 

□ B 
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te foregoing is too simple to describe the 
[:'* of your institution, please indicate if 
d-modal" curve exists or what other 
ation exists. Comment. 
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APPENDIX B 



Software Cost Estimation Procedure Data Form 



The purpose of this q uestionnaire is to arrive at an 
estiinate of the cost of the software being used with the media 
system under study. Making such an estimate is q uite diffi- 
cult. This is due to the number of factors involved in estimating 
the cost of software and the difficulty of placing a price tag on 
each of the factors involved in the estimate. The following set 
of q uestions is designed to gather information about the factors 
involved in the cost of software, then to meaningfully combine 
these factors to arrive at the final cost estimates of software: 

SOF'fWARETYPE 

1. Which of the following types of software materials are used 
with this system? 

Prepared on the campus Q % 

Commercially prepared (purchased) % 

If both campus prepared and commercially prepared software 
materials are used, please indicate an estimate of the percentage 
of the total software which is campus prepared and the percentage 
which is commercially prepared on the line next to the choices 
provided. If you are using commercially prepared software only, 
skip to Question 13. If you are using campus prepared materials 
go to Question 2, beginning immediately below. 

ON-CAMPUS PREPARED MATERIALS 

In this series of questions you are asked to answer questions 
about and calculate the cost of the on- campus prepared materials 
in your software library. 

For each of the materials media in your library prepared 
on- campus, you will be asked to calculate cost as follows. 



Calculate the gross cost of each of the materials in your software 
liorary, include professional time required to prepare and test the materials, 
technician time, cost of materials (tape,, film, etc.) and cost of hardware 
time required to test and revise software and miscellaneous cost. 

2. This question deals with the medium on which materials are stored in 
your library of software. Plea.se look at the diagram below and check off 
in the boxes to the right those storage media which are being used for the 
software materials contained in your library. 



SOFTWARE MATERIALS MEDIA 

Number Average Cost 

Medium On Hand Per U nit 

Microfiche Cards 
Film Strips 

Transparencies 

Slides (2x2, 10x10, other) 

(lantern) 

Books 
Worksheets 
Panels/charts /Maps 
Games 

Tape (1/4" Audio, Video) 

Oxide Discs 
Magnetic Stripe Films 
Optical Sound Films Edge 
Records 

Belts and other Audio Sources 

Lectures, Live 

Objects 

Kits 

8 mm Loop 
VTR 

Others 
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3. Indicate the size of the library of software materials used with this system, 
or each of the media specified in Question #1. For each specify a measure of 

the physical length of the software materials. For example, indicate for slides 
the number of slides; for film, the length of film; for books, the number of 

pages, etc. Also, Indicate the length of instructional time for which these 

software materials are used during one academic year. 

4. This question attempts to determine the size of your software library in 
terms of the number of copies in each medium. For each of the media checked 
Cbbove, please indicate the gross number of copies in your software library. 
Then indicate the net number of copies which have been in use during the last 

year. 

5. In this question you are asked to estimate the average frequency of use of 
each of the media in your software library. Indicate the average frequency of 
use of each of the media in terms of number of times used per academic year. 

6. Please indicate here the gross number of student hours per academic 
year of use for each medium. 



7. In this question you are asked to begin to provide the factors which 
make up on-campus prepared software costs. Please indicate for each 
medium the professional cost of software materials preparation (Hours X 

Salary/hours). 

8. Please indicate the cost of technicians' time in preparing these software 
materials. (Hours X Salary/hour) 
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9. 



Please indicate the cost of materials for this software. 



10. Please indicate the cost of hardware time to test and revise software. 

11. Please indicate any miscellaneous cost. 

12. Next estimate the life expectancy of each of the software materials in your 
library. 

Based on these cost figures an estimate of the cost of software materials 
in your library on a yearly basis can be made. Then the cost per student 
hour of use per academic year can be calculated for each media, dividing the 
total cost per academic year by the gross number of stiident hours of use 
per year. 

If you are also using commercially prepared materials, please go on 
to the next section. 
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COMMERCIALLY PREPARED MATERIALS 

In this series of questions you are asked to provide information about 
and calculate the cost of each of the commercially prepared materials in your 

software library. 

13. This ciucsUoi, deals with the medium on which materials are stored in 
your library of software. Please look at the diagram below and check off 
in the boxes to the right those storage media which arc being used for the 
software materials contained in your library. 



SOFTWARE MATERIALS MEDIA 

Number 

Medium On Hand 



Average Cost 
Per Unit 



Microfiche Cards 
Film Strips 
Transparencies 
Slides (Zx2, 10x10, other) 
(lantern) 

Books 
Worksheets 
Panels/charts /Maps 
Games 

Tape (I/4" Audio, Video) 

Oxide Discs 
Magnetic Stripe Films 
Optical Sound Films Edge 
Records 

Belts and other Audio Sources 

Lectures, Live 

Objects 

Kits 

8 mm Loop 
VTR 

Others 
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14. Indicate the sii-.e of the library of software materials used with this system, 
for each of the media specified in Quesaon 111. For each specify a measure of 
the physical length of the software materials. For exampJc'. indicate for slides 
the number of slides; for film, the length of film: for books, the number of 
pages, etc. Also, indicate the length of instructional time for which those 
software materials are used during one academic year. 

15. This question attempts to determine the size of your software library m 
terms of the number of copies in each medium. For each of the media checked 
above, please indicate the gross number of copies in your software library. 
Then indicate the net number of copies which have been in use during the last 

year. 

16. In this question you are asked to estimate the average frequency of use of 
each of the media in your software library. Indicate the average frequency of 
use of each of the media in terms of number of times used per academic year. 

17. Please indicate here the gross number of student hours per academic 
year of use for each medium. 



18 . Calculate the gross cos 



t of the software materials including 



the initial purchase cost, and shipping or handling cost for each 



media. 



it estimate the life expectancy of . each of the software 



istimate of the cost of software 



19 . Next 

materials in your library. 

Based on these cost figures, an ci 
material in your library on a yearly basis can be made. Ihe cost 
per student hour of use per academic year , can be calculated by dividing 
the total cost per academic year by the gross number of student hours 




of use per year. 
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